Critical Appraisal of Scientific Literature

Critical Appraisal of Scientific Literature

André Valdez, PhD

Stanford Health Care

Stanford University School of Medicine

Confidential – For Discussion Purposes Only

Learning Objectives

Understand the major components of a scientific article

Learn about some of the most common study designs and analyses

Become more familiar with literature appraisal tools

2


Why Does This Matter?

Greater understanding of the literature = better advocacy for EBP

Increased literacy on a given clinical topic and gaps in knowledge

Increased awareness of a given study’s potential and its limitations

Grounding of clinical decision making in sound scientific reasoning

3


Sections of a Scientific Article – Abstract

Introduction

Objectives/hypothesis

Methods

Results

Discussion/Conclusion

Two general types: structured and unstructured

− Structured contains headings for each major section of a study

− Unstructured does not contain headings

4


Sections of a Scientific Article – Introduction

Helps the reader become familiar with the topic

Refers to recent literature and clearly maps out study’s rationale

Supports central argument with citations

What question(s) does the study attempt to answer?

Why is the study design appropriate?

5

(duPrel et al. 2009)


Sections of a Scientific Article – Methods

Highlights data for appraisal of study’s validity

Describes all phases of planning, study sample, procedures, and statistical tests

Lays out study design, which is paramount

Precise description of inclusion/exclusion criteria

Thoroughly describes how data were collected, measurement scales of

variables, etc.

Sample size calculation

6



Sections of a Scientific Article – Results

Clear and objective presentation of findings

Summary statistics on all variables

Inferential statistics on variables of interest

Tables and figures to improve clarity

7



Sections of a Scientific Article – Discussion

Compares findings with what’s currently known

What has the study added to the existing body of knowledge?

What conclusions may the reader draw?

Do the interpretations follow logically from the results?

Analyzes study limitations

Statistical significance not the same as clinical relevance

8



Common Study Designs

Quality Improvement/Process/Performance Evaluation

− Set up to enhance clinical practice at a local level (not generalized)

− Often but not always quasi/non-experimental (no random assignment)

Uncontrolled pre-post comparison

Controlled pre-post comparison

Qualitative experience

Cost-effectiveness

Human Subject Research

− Set up to generate new knowledge (generalizable)

− Often but not always experimental (random assignment)

Randomized controlled trial

Quasi/non-experimental/observational

− Case-control

− Cohort

− Longitudinal

− Retrospective chart review

(Portela et al. 2015; Topics in Biostatistics 2007)

(Portela et al. 2015)

9


Common Statistical Analyses

Variable scale: continuous (parametric: assumptions about data distribution)

− Independent-samples t-test

− Paired t-test

− Analysis of Variance

− Linear regression

Variable scale: categorical

− Chi-square test

− Logistic regression

Variable scale: ordinal (non-parametric: distribution free)

− Signed-rank test

− Rank-sum test

− Kruskal-Wallis test

(http://stattrek.com/)

10

http://stattrek.com/

http://stattrek.com/


Literature Appraisal Tools – Johns Hopkins

11

Evidence Levels

Quality Guides

Level I Experimental study, randomized controlled trial (RCT) Systematic review of RCTs, with or without meta-analysis

A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control; definitive conclusions; consistent recommendations based on comprehensive literature review that includes thorough reference to scientific evidence

B Good quality: Reasonably consistent results; sufficient sample size for

the study design; some control, fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review that includes some reference to scientific evidence

C Low quality or major flaws: Little evidence with inconsistent results;

insufficient sample size for the study design; conclusions cannot be drawn

Level II Quasi-experimental study Systematic review of a combination of RCTs and quasi-experimental, or quasi-experimental studies only, with or without meta-analysis Level III Non-experimental study Systematic review of a combination of RCTs, quasi-experimental and non-experimental studies, or non-experimental studies only, with or without meta-analysis Qualitative study or systematic review with or without a meta-synthesis

Evidence Levels & Quality Guides

(http://www.hopkinsmedicine.org/evidence-based-practice/jhn_ebp.html)

http://www.hopkinsmedicine.org/evidence-based-practice/jhn_ebp.html








Literature Appraisal Tools – Johns Hopkins

12

Article

# Author &

Date Evidence

Type

Sample, Sample Size &

Setting

Study findings that help answer the EBP question

Limitations Evidence Level & Quality

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Evidence Summary Table

(http://www.hopkinsmedicine.org/evidence-based-practice/jhn_ebp.html)









Literature Appraisal Tools – STROBE

13

(http://www.strobe-statement.org/index.php?id=available-checklists)

http://www.strobe-statement.org/index.php?id=available-checklists









14

✔

✔ ✔

✔

✔

✔











15

✔

✔

✔

✔










Summary

Understanding and appraising scientific literature is crucial for effective EBP

Learning about study designs and analyses will help you interpret findings

Using appraisal tools is a good way to organize your literature review

Critical appraisal promotes critical thinking about your topic

16

Critical Appraisal of Scientific Literature

Documents

Transcript of Critical Appraisal of Scientific Literature