Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology...
-
date post
18-Dec-2015 -
Category
Documents
-
view
220 -
download
4
Transcript of Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology...
![Page 1: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/1.jpg)
Brian L. Strom, M.D., M.P.H.Chair and Professor, Department of
Biostatistics and Epidemiology
Director, Center for Clinical Epidemiologyand Biostatistics
George S. Pepper Professor of Public Health and Preventive Medicine
Professor of Biostatistics and Epidemiology, Medicine, and Pharmacology
University of PennsylvaniaSchool of Medicine
CCEB
![Page 2: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/2.jpg)
CCEB
“Sampling”
1. What is an appropriate sample size of respondents to best determine risk of sound and look alike proprietary names in the prescription drug study group? In a focus group? In a survey document?
2. Should sample be randomly selected? Is it important to have a statistical significance for this type of evaluation?
![Page 3: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/3.jpg)
![Page 4: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/4.jpg)
CCEB
“Sampling”
• Introduction
• Very general principles of sample size calculations and sampling
• Application of general principles to this situation
• Recommendations for research, to guide the future
![Page 5: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/5.jpg)
CCEB
Options in Research Design
• Analytic Studies– Experimental Study– Prospective Cohort Study– Retrospective Cohort
Study– Case-Control Study
• Descriptive Studies– Analyses of Secular
Trends– Case Series– Case Reports
![Page 6: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/6.jpg)
Case-Control StudiesCase-Control Studies
DiseaseDisease
Coh
ort
Stu
die
sC
oh
ort
Stu
die
s
Fact
or
Fact
or
PresentPresent(cases)(cases)
AbsentAbsent(not exposed)(not exposed)
PresentPresent(exposed)(exposed)
AbsentAbsent(controls)(controls)
AA
DDCC
BB
![Page 7: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/7.jpg)
CCEB
Determinants of Needed
Sample Size• Alpha
• Beta
• Variability (SD)
• Delta--how small a difference do you want to be able to detect?
![Page 8: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/8.jpg)
CCEB
Determinants of NeededSample Size: Cohort
Study• Alpha
• Beta
• Incidence in the unexposed control group
• Delta--how small a RR do you want to be able to detect?
![Page 9: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/9.jpg)
CCEB
Determinants of Needed Sample
Size: Case-Control Study
• Alpha
• Beta
• Prevalence in the undiseased control group
• Delta--how small a RR do you want to be able to detect?
![Page 10: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/10.jpg)
CCEB
Sampling
Study Sample
Conclusion About a Population(Association)
Conclusion About Scientific Theory(Causation)
Statistical Inference
Biological Inference
![Page 11: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/11.jpg)
CCEB
Application of GeneralPrinciples to This
Situation
• THE Central Principleof Research Design: The question is, what is the question??
![Page 12: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/12.jpg)
CCEB
Application of GeneralPrinciples to This
Situation• In this situation, there are no
a priori hypotheses being tested to be able to consider sample size calculations or questions of sampling• What is being performed is
essentially qualitative research
![Page 13: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/13.jpg)
CCEB
Recommendations for Research,
to Guide the Future
**Evaluate the current process in a quantitative fashion**
![Page 14: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/14.jpg)
CCEB
General Strategy for Evaluation
• Standardize procedure• Test for reliability/
reproducibility• Test for validity• Make changes in the
procedure accordingly
![Page 15: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/15.jpg)
CCEB
Standardize Procedure
• Choose among the current possible approaches a “standard” to be evaluated more rigorously
![Page 16: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/16.jpg)
CCEB
Test for Reliability/Reproducibility
• Evaluate the same drug names in the same process with multiple different groups of survey prescribers and experts, to look for whether there is adequate agreement• If no reliability, validity is
impossible and procedure should be abandoned
![Page 17: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/17.jpg)
CCEB
Test for Validity
• Gold standard needed:–Drug names rejected in initial FDA review
–Drug names withdrawn due to problems
–Direct measurement of error rate
![Page 18: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/18.jpg)
CCEB
Retrospective: Drug NamesRejected in Initial FDA
Review
• Problem: was the initial review decision correct?
![Page 19: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/19.jpg)
CCEB
Retrospective: Drug NamesWithdrawn Due to
Problems
• Knowledge of reviewers could be problematic–Other countries–Years ago, and using new
pharmacists
• Was the withdrawal decision a correct one?
![Page 20: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/20.jpg)
CCEB
Prospective: DirectMeasurement of Error Rate
• Simulate real life situation in a study setting
• Choose good and bad options for new names
• Enter possible new names into prescription entry computer program
• Ask large numbers of docs to write orders
• Ask large numbers of pharmacists to “fill” each rx, entering into prescription entry computer program
• Directly measure resulting error rate
![Page 21: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/21.jpg)
CCEB
Make Changes in theProcedure
Accordingly• Determine appropriate cutpoint
for expert ratings, ROC curve vs. gold standard• Determine appropriate sample
sizes through simulation, ie how many are needed to achieve results consistent with the gold standard• Modify processes accordingly
![Page 22: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/22.jpg)
CCEB
Potential Sources of Support
• FDA extramural funds
• AHRQ patient safety funds
• NIA pharmacology program
• NIGMS pharmacology program
![Page 23: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/23.jpg)
CCEB
Conclusions
• Applying a quantitative approach to evaluating what has so far been a qualitative one, could lead to major changes in the procedure, and major improvements in the net results
![Page 24: Brian L. Strom, M.D., M.P.H. Chair and Professor, Department of Biostatistics and Epidemiology Director, Center for Clinical Epidemiology and Biostatistics.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649d225503460f949f89ac/html5/thumbnails/24.jpg)
Statitsically incorrect