www.monash.edu.au
Interpreting interventional studies
Dr Paul JenningsDepartment of Community Emergency Health and Paramedic PracticeMonash University
Interpreting an interventional study
• Interventional study– Investigators assign an exposure
Types of Quantitative
study designs
Experimental / Interventional
Randomised Controlled Trial (RCT)
Non / Quasi-randomised
Controlled Trial
Observational
Cohort (Longitudinal)
Case-control
Cross-sectional
Ecological (population-
based)
Dissemination of findings?
• Abstract• Introduction• Methods• Results• Discussion/Conclusion• References• Figures/Tables
Components of a paper?
• Introduction• Methods• Results• and• Discussion/Conclusion
IMRaD
• Title: Does it convey a message?
• Authors: Are they experts in the field/study type?
• Keywords: Appropriate, MeSH (medical subject headings).
• Abstract: Succinct overview of the paper, reflects manuscript?
• Publication Date: Recent? Still Relevant?
Preliminary Information
• The informative preview section of the paper.• What is the papers purpose?• Why was it done?• How was it done? Setting/design type/methods• What were the main findings?• What do these findings mean and why is it
important?• Structured or unstructured• Usually up to 250 words.
Abstract
Introduction
• Provides the context of the paper.• Describes what is known about the topic.• Describes gaps and limitations in the
literature.• Describes and introduces the
question/problem to be addressed.• States the rationale/purpose of the paper.
• Setting:– Where did the study occur?
• Participants:– Number, characteristics (homogeneity of
groups), selection/randomisation process.– Drop-out or exclusions
• Procedures:– Step-by-step process– Can the study be replicated?– Operational definitions provided
Methods & Design
• Are the methods used reliable?• Are the methods used valid?
– Randomisation? Process used (True vs psuedo)
– Was allocation concealed?– Analysis of groups (ITT vs per-protocol)– Blinding?– Group treatment (same?)– Representativeness
Methods & Design
• What was measured?• Does it report descriptive or inferential
statistics?• Are the statistical methods appropriate?• Is the significance level set and accurately
reported?• Are confidence intervals reported?• Does study power need to be reported?
Data Analysis
• Are the results clearly presented (text, tables, figures)?
• Does each result have a method?• Is there no discussion of results?• Is the precision of the results presented?
– 95% Confidence intervals
• Table 1
Resultsblah blah
blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah
blah
blah blah blah blah blah blah blah blah blah blah
• Revision of the research question.• Presentation of the important results.• No new results presented.• Discussion of the results in the context of the
available literature.• Are there strengths/limitations discussed?• What is the generalizability of the results?• Recommendations for future research.• Can these findings be applied to patient care?
Discussion
• Are the conclusions supported by the results and available evidence?
• Are they justified?• Are they proportionate to the methods,
sample and design used? (Consider the NHMRC levels of evidence).
Conclusion
• Are they published accurately with full details• Does the list contain both contemporary and
seminal literature?• Any key papers missing?
References
• Many developed.• Example: Downs & Black. J Epidemiol
Community Health. 1998; 52:377-384.• Five domains:
– Reporting: 10 questions– External validity: 3 questions– Bias: 7 questions– Confounding: 6 questions– Power: 1 question (power to detect a clinically
important difference)
Review Checklists
Reporting
External Validity
Bias
Confounding
• CONSORT (CONsolidated Standards of Reporting Trials):
– Useful guide for reviewing RCTs– http://www.consort-statement.org/
• STROBE (Strengthening the Reporting of Observational studies in Epidemiology):
– Useful guide for reviewing observational studies.– cohort, case-control, and cross-sectional studies – http://www.strobe-statement.org/
Paper Presentation Guides
• There are few, if any, “perfect” studies/papers.• All papers have been through peer review.• Does the paper have a clear aim?• Is the method appropriate to answer the aim?• Does each result have a method?• Are the results adequately discussed in context?• Are the study limitations addressed?• Are the conclusions appropriate?
Summary
• Coughlan M, Cronin P, Ryan F. Step-by-step guide to critiquing research. Part 1: quantitative research. Br J Nurs. 2007; 16(11):658-663.
• Stockhausen L, Conrick M. Making sense of research: a guide for critiquing a paper. Contemp Nurse. 2002; 14(1):38-45.
Useful References
Thank-you for your attentionQuestions?
Top Related