Appraising Randomized Clinical Trials and Systematic Reviews
October 12, 2012
Mary H. Palmer, PhD, RN, C, FAAN, AGSF
University of North Carolina at Chapel Hill
Objectives
• Discuss systematic reviews in the context of the pyramid of evidence
• Describe the importance of the CONSORT statement for randomized controlled trials
• Appraise select randomized controlled trials• Appraise select systematic literature reviews • Describe the process of synthesis and
evaluation
Appraising Systematic Reviews
Systematic reviews are a compilation of similar studies that address a specific clinical question.
Systematic reviews are not the same as a literature review or narrative review
The process the was used to conduce the review should be explicit
Look for a detailed description of the databases accessed, the search strategies, and the search terms
Clinicians should be able to clearly see which studies were included and which were excluded and how selected studies were assessed
Systematic Reviews• A specific research question is answered.• Detailed inclusion and exclusion criteria are
explicated.• Elaborate and thorough search strategies.• Standardized review protocols that use trained
reviewers (rather than PIs)• Standardized abstracting process for capturing
details• Pre-established quality criteria to rate the value
of the individual studies.
Systematic Reviews
• How is bias eliminated?– Databases accessed– Search strategies used– Search terms used– Years searched – Language used – Unpublished studies included (publication bias)
Systematic Reviews
Randomized controlled trialsPRISMA
Meta-analyses and observational and studiesMOOSE
Meta-Analysis
• Meta-analysis is a systematic review, not all systematic reviews are meta-analysis.
• Quality of studies included is critical. • Meta-analyses pool data from individual
studies– Increased sample size – Pooled statistical results – Definitive conclusions possible
Meta-Analysis (cont’d) Remember, the report should inform clinicians about how
data were extracted from the individual studies Summary Meta-analysis: the statistical approach to synthesizing the
results of two or more studies A relatively new methodology that has become a
hallmark of EBP
As with all methodologies, applicability must be considered
CONSORThttp://www.consort-statement.org/consort-statement/translations/
CONSORT Translations
•The CONSORT website will announce and host translated CONSORT documents that we are aware of. If you are interested in translating a CONSORT document into another language, please see the CONSORT translation policy.
•CONSORT 2010 Translations•Chinese* •CONSORT Statement PDFCONSORT Explanation & Elaboration PDFChecklist PDF / MS WordFlow Diagram PDF / MS Word
Appraising Randomized Controlled Trials (RCTs)
These studies can accurately establish cause and effect and can inform the efficacy and effectiveness of interventions
Questions related to validity of RCTs Were the subjects randomly assigned to the
experimental and control groups? Was random assignment concealed from the
individuals who were first enrolling subjects into the study?
Were the subjects and providers kept blind to study group?
Appraising the Validity of RCTs
Were reasons given to explain why subjects did not complete the study?
Were the follow-up assessments long enough to fully study the effects of the intervention?
Were the subjects analyzed in the group to which they were randomly assigned?
Was the control group appropriate? Were instruments valid and reliable? Were the subjects in each of the groups similar on
demographic and baseline clinical variables?
Evaluation and Synthesis of Quantitative Studies
Once studies have been selected, they should be melded together into a synthesis upon which to base practice and standards
Individual studies should be evaluated using an evaluation table - see Table 5-8 in your textbook
Synthesis is NOT reporting the findings of study after study; it is combining, contrasting, and interpreting a body of evidence to reach a conclusion
Question
Tell whether the following statement is true or false. Meta-analysis results in evidence that is applicable to a
larger patient population than individual studies.
Answer
False Rationale: In meta-analyses, combining the results of
several studies produces a larger sample size and thus greater power to accurately determine the magnitude of the effect. This does not, however, increase the size of the relevant patient population.
Homework
Find a systematic review or a randomized controlled trial related to your PICOT question.•Write a critical appraisal and send it and a pdf of your article to:
Deadline: October 22, 2012
Top Related