Systematic Review Module 3: Study Eligibility Criteria
description
Transcript of Systematic Review Module 3: Study Eligibility Criteria
Systematic Review Module 3: Systematic Review Module 3: Study Eligibility CriteriaStudy Eligibility Criteria
Melissa McPheeters, PhD, MPH Melissa McPheeters, PhD, MPH Associate Director Associate Director
Vanderbilt University Evidence-based Practice CenterVanderbilt University Evidence-based Practice Center
Learning ObjectivesLearning Objectives
To understand the role of selection criteria in framing a systematic review
To know when and how to set selection criteria
To understand the effect of selection criteria on interpretation of a review
2
CER Process OverviewCER Process Overview
Prepare topic:
· Refine key questions
· Develop analytic frameworks
Search for and select
studies:
· Identify eligibility criteria
· Search for relevant studies
· Select evidence for inclusion
Abstract data:
· Extract evidence from studies
· Construct evidence tables
Analyze and synthesize data:
· Assess quality of studies
· Assess applicability of studies
· Apply qualitative methods
· Apply quantitative methods (meta-analyses)
· Rate the strength of a body of evidence
Present findings
3
Study Selection CriteriaStudy Selection Criteria
Function the same in systematic reviews as in primary research
Should reflect the analytic framework and key questions
Are powerful tools for widening or narrowing the scope of a review
Provide information to determine whether reviews can be compared or combined
4
Some Example CriteriaSome Example Criteria
Adult, community-dwelling females Study of a screening tool for depression United States only Hospital-based studies only N > 200 Randomized controlled trials
5
Using Broad CriteriaUsing Broad Criteria
Can be as broad as all related studies Helpful for exploring “what is known” May result in too much literature to
feasibly review or disparate literature that cannot be compared
6
Using Narrow CriteriaUsing Narrow Criteria
May return too little literature Can result in inability to answer the
intended question Helpful in culling homogenous literature Can reduce size of the literature to a
manageable scope
7
Bias in this ContextBias in this Context
Distortion of the estimate of effect that comes from how studies are selected for inclusion
Affects the applicability or “external validity” of the review itself
8
Examples of Bias in this ContextExamples of Bias in this Context
Included studies may not have been conducted in the patient population whose care the review is intended to affect– e.g., the use of studies of twin pregnancies
in a review of preterm labor management for low-risk women
Selection criteria may be set to include more of a certain study type that either overestimates or underestimates effectiveness
9
Selecting CriteriaSelecting Criteria
Review study goals Assess analytic framework and key
questions Tie criteria to PICOTS Set criteria before beginning abstract
review
10
Basic Questions Basic Questions
What is the relevant population?What is the relevant population? What is the intervention of interest?What is the intervention of interest? To what exposure is the intervention To what exposure is the intervention
being compared? being compared? What outcomes are relevant?What outcomes are relevant? Should time to outcome be limited?Should time to outcome be limited? In what setting should the results be In what setting should the results be
applicable?applicable?
11
Exercise 1Exercise 1
What would you do if you were asked to review the literature on transition support for adolescents with autism spectrum disorders (ASD) entering adulthood?
Before seeing the key questions, consider the categories of criteria that we will want to apply.
12
Apply PICOTSApply PICOTS
Population—condition, disease severity and stage, comorbidities, patient demographics
Intervention—dosage, frequency, method of administration
Comparators—placebo, usual care, or active control
Outcomes—health outcomes, morbidity, mortality, quality of life (QoL)
Timing—Duration of followup Setting—Primary, specialty, in-patient,
cointerventions
13
PICOTSPICOTS
Population
Intervention Comparators
Outcomes
Timing
Setting
What constitutes an adolescent? What constitutes a diagnosis of ASD?
How is transition support defined? Do we compare to no transition support or
directly compare types of support? What are the goals for adolescents with ASD
as they transition to adulthood? Should they be individually focused?
How quickly should the outcomes be apparent?
Is transition support provided in multiple settings, such as schools, clinics, and the community?
14
What Would You Do with…What Would You Do with…
A paper that was about “individuals over age 10”?
A paper that was about an intervention for individuals with a range of developmental disabilities?
Or, conversely, a paper that was specifically about children with Asperger’s syndrome but not other ASDs?
15
Example of a Narrow ScopeExample of a Narrow Scope
What is the efficacy of home uterine activity monitoring for preventing preterm birth among women at low risk of a preterm birth?
16
Implications of a Question Implications of a Question with Narrow Scopewith Narrow Scope
Efficacy: RCTs only Low risk: no prior preterm birth No. of studies: 11
17
Overactive Bladder StudyOveractive Bladder Study
Management of OAB among women Considerations
– OAB is a fairly difficult condition to define– Treatments include pharmacologic,
behavioral, complementary and alternative medicine (CAM) and procedural—and each area includes very different types of studies
– Study of OAB is often combined with other types of urological conditions, such as stress incontinence or prostate issues
18
Exercise 2Exercise 2
Set two criteria and consider the Set two criteria and consider the potential implicationspotential implications– Minimum study size
– Gender of study participants
19
Study SizeStudy Size
50 at study start Implications
– Excluded for size only: 79
– Excluded for N < 20: 36
– Excluded for N 20 to 29: 23
– Excluded for N 30 to 39: 8
– Excluded for N 40 to 49: 12
20
GenderGender
Studies had to include at least 75% women
This decision was based on expert opinion and the size and scope of the literature
40 studies were excluded with less than 75% women
27 additional studies would have been excluded had the review been limited to studies of only women
21
Other ConsiderationsOther Considerations
What study designs should be What study designs should be included?included?
Include foreign studies? Other Include foreign studies? Other languages? Studies conducted in languages? Studies conducted in specific parts of the world?specific parts of the world?
Include “grey” or “fugitive” literature?Include “grey” or “fugitive” literature?
22
Types of Studies?Types of Studies?
Limit to RCTs? Include observational studies?
– If so, which kinds?
What is the value of a case series? What counts as a case series?
23
Example of Maternal-fetal Surgery Technical Brief
Included case series with N ≥ 2 Only 3 of 169 studies were RCTs, and
122 were case series Because of the relative newness of this
area of research, it was important to capture data even from studies without comparison groups
24
Observational StudiesObservational Studies
Types– Cohorts (with comparisons)
– Case controls
– Case series
– Registries/databases
25
Observational StudiesObservational Studies
Well-done observational studies can address issues of applicability and the need for longer-term outcomes if they:– Include more representative patient
populations– Have relevant comparators– Report more meaningful clinical outcomes
over longer time frames Observational studies may be a better
source of information about harms
26
Foreign LiteratureForeign Literature
Positive findings may be more likely to be published in high-profile journals published in English
Therefore, to include only English-language journals may overestimate the positive effect of an intervention
Empirically, the bias associated with limiting one’s review to English has been shown to be small (Moher et al., 2000; Gregoire et al., 1995)
27
Systematic Review on Cesarean Systematic Review on Cesarean DeliveryDelivery
Systematic review on outcomes of cesarean delivery on maternal request
Conducted for the National Institutes of Health-Office of Medical Applications of Research State-of-the-Science conference
28
Exercise 3Exercise 3
Define the appropriate population group and comparator.
What other limitations would you put on included literature?
29
The ChallengeThe Challenge
No evidence on outcomes of CDMR vs. other modes of delivery
Urgent need for actionable evidence Need to recognize and account for
confounders
30
SolutionSolution
Expand search to include proxies Weight rungs of evidence to account for
confounding– Highest rung: Trials of breech delivery, but only for
maternal outcomes
– Next rung: Planned cesarean vs. planned vaginal
– Lowest rung: Comparisons of maternal and neonatal outcomes from actual modes of delivery
31
SummarySummary
Selection criteria are essential for setting Selection criteria are essential for setting the scope of the reviewthe scope of the review
They should be tied to the analytic They should be tied to the analytic framework, key questions, and PICOTS framework, key questions, and PICOTS
When properly applied, selection criteria When properly applied, selection criteria can reduce bias and support the can reduce bias and support the applicability of the reviewapplicability of the review
32