Why read papers: An Introduction to Evidence

Based Medicine

Arash Etemadi, MD PhDDepartment of Epidemiology and

Biostatistics,School of Public Health, Tehran University of

Medical Sciencesaetemadi@tums.ac.ir

Why do we read articles?

• Browsing

• For information

• For research

• For review

• A paper from someone we know

• For going to sleep!

Do We Read ?

• Self-reported reading time per week. (University setting)– Medical students 60 min.

– Interns none

– Senior residents 10 min.

– Fellows 45 min.

– Attendings graduating

• Post 1975 60 min.

• Pre 1975 30 min.

Do We Read?

• University of Virginia

• Mailing to primary care physicians– 50% had not read a medical journal article in

the last year.– The most commonly sited source of

information was pharmacutical representatives.

Why Don’t We Read ?

• We’re lazy? – The fact of the matter is that none of us likes

feeling out of date. We like it so little in fact that we are willing to work at night and on weekends in an effort to stay current.

• Frustration.– Conflicting information– No one taught us HOW or WHAT to read.

What evidence-based medicine is:

the integration of best research evidence

with clinical expertise

and patient values.

Is EBM …

reading articles?

I. Best Research Evidence:

• clinically relevant research, often from the basic sciences of medicine, but especially from patient centered clinical research.

• Has a short doubling-time.• Replaces currently accepted diagnostic tests

and treatments with new ones that are more powerful, more accurate, more efficacious, and safer.

II. Clinical Expertise:

• rapidly identify each patient's unique health state and diagnosis,

• their individual risks and benefits of potential interventions,

• and their personal values and expectations.

III. Patients’ Values & Expectations

• the unique preferences, concerns and expectations each patient brings to a clinical encounter

• must be integrated into clinical decisions if they are to serve the patient.

A model for evidence-based clinical A model for evidence-based clinical decisionsdecisions

Traditional medicine

experiencesPathophysiology,

references,…

Patient value

Practice Based

On Theory

Retrolental Fibroplasia Lesson(1)(Silverman, 1977, Scientific American)

• In early 1940’s, epidemic of retrolental fibroplasia began in premature infants

Case reviews indicated “state of the art” care, including high concentration of O2

• Suspicion arose that cause occurred after birth resulting in progressive changes in retina blood vessels

• Early 1950’s, ACTH treatment proposed and RCT tested

Arm Results

ACTH 1/3 Blind

vs. ACTH = adrenocorticotrophichormone

Placebo 1/5 Blind

Retrolental Fibroplasia Lesson(2) (Silverman, 1977, Scientific American)

• Search for a cause– High dose O2 suspected based on anecdotal evidence of

147 infants

– Another observational series of 479 infants claimed benefit

• One study attempted to lower O2 dose– But nurses would turn O2 on at night and off in a.m.

– Felt no or low O2 unethical

Retrolental Fibroplasia Lesson(3)(Silverman, 1977, Scientific American)

• 1953 NIH Conference - Two opinions1. Need controlled study2. No need, O2 already convicted

• 1953 RCT began on 800 infants % Blinded

- standard O2 dose 23%

- 50% O2 dose only for clinical indications7%

Also found a dose response

• 1954 Results published, high O2 practice stopped and epidemic subsided

However, not before 10,000 infants had been blinded

Failure to Use Therapy Based on Theory

Chronic Heart Failure

• Not many good therapies in 1980’s

• Beta blockers known to be effective in post MI patient care– Reduces mortality– Lowers blood pressure– Slows and regulates heart rate

• Proscribed for heart failure patients

Beta-Blocker HFTrial Features

• Class II-IV heart failure

• Low ejection fraction

• Beta-blocker vs. placebo

• Randomized double blind

• Several thousand patients

MERITTotal Mortality

CIBIS-II

Lancet, 1999

COPERNICUS

NEJM, 2001

Long Standing Treatment

Based on Theory and

Observation/Association

• Hypothesis that HRT reduced coronary heart disease

• Supportive data– Lipid lowering– Non-human primate studies– Observational studies

Hormone Replacement Therapy (HRT)

Observational Studies

• Example – Refs:1) Stampfer & Coldiz (Prev Med 1991)

Nurses Health Study

2) Grady (Ann Int Med 1992)

3) Cauley, Cummings, et al. (Am J OB/GYN, 1990)

4) Grodstein, Stempfer, Manson (NEJM 1996)

• Suggest 40-50% reduction in CHD risk

HRT POPULAR

• 1/3 of post-menopausal women use HRT

• Second most prescribed drugs

• Year 2000, 46 million prescriptions for Premarin (Estrogen)

• $1 billion in sales

• 22 million prescriptions for PremPro (E+P)

HORMONE REPLACEMENT THERAPY FOR POSTMENOPAUSAL

WOMEN• Secondary Prevention

HERS: Hully S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, Vittinghoff E; for the HERS Research Group: Randomized trial of estrogen plus progestin for secondary prevention of coronary

heart disease in postmenopausal women. JAMA 28(7):605-13, 1998.

• Primary PreventionWHI: Writing Group for the Women’s Health Initiative Investigators: Risks and benefits of estrogen plus progestin in healthy postmenopausal women. Principal results from the Women’s Health Initiative Randomized Controlled trial. JAMA 288:321-333, 2002.

HERS JAMA 28(7):605-13, 1998

• Postmenopausal women• Secondary prevention, patients had

documented cardiovascular disease• Estrogen-progestin vs. placebo• Randomized double blind• Outcomes

– CVD mortality

– Fractures

HERS• Observed early clotting problems

– DVTs– PEs

• Fracture trend for benefit

• Early negative trend in mortality that reverses to neutrality (non-definitive)

HERS MORTALITY

JAMA, 1998

HERS IMPACT

• Many believed results only applied in secondary prevention

• Many interpreted trend reversal as suggesting benefit if longer follow-up

• No perceptible impact on HRT use since HRT has other benefits

WOMEN’S HEALTH INITIATIVE

JAMA 288(3):321-33, 2002• A large factorial trial evaluating HRT, low

fat diet and calcium

• Multiple outcomes for each treatment

• For HRT– Coronary heart disease (MI & CHD death)– Invasive breast cancer– Global index– Fractures

WHI373,092 Women Initiated Screening

18,845 Provided Consent & Reported No Hysterectomy

8506 Assigned to Receive Estrogen + Progestin

8102 Assigned to Receive Placebo

16,608 Randomized

Status on April 30, 2002

7968 Alive & Outcomes Data Submitted in Last 18 Months307 Unknown Vital Status231 Deceased

Status on April 30, 2002

7608 Alive & Outcomes Data Submitted in Last 18 Months276 Unknown Vital Status218 Deceased

WHI

Cumulative Dropout and Drop-in Rates by Randomization Assignment and Follow-up Duration

JAMA, 2002

Wisconsin State Journal2002

WHI

JAMA, 2002

Kaplan-Meier Estimates of Cumulative Hazards for Selected Clinical Outcomes

WHI

JAMA, 2002

Kaplan-Meier Estimates of Cumulative Hazards for Global Index and Death

WHIKaplan-Meier Estimates of Cumulative Hazards

for Selected Clinical Outcomes

JAMA, 2002

WHIKaplan-Meier Estimates of Cumulative Hazards

for Selected Clinical Outcomes

JAMA, 2002

HRT: Low But Increased Risk

Rate %HR

Outcome HRT PLBO

CHD .37 .30 1.29

Stroke .29 .21 1.41

DVT .26 .13 2.07

PE .16 .08 2.13

Breast CA .38 .30 1.26

Death .52 .53 .98

Ann of Int Med 137(4), 2002

Evidence Based Medicine

• For important questions with serious mortality/morbidity, need stronger evidence such as RCTs

• If RCTs not possible, need to be cautious & vigilant about Treatments only based on observation/association or theory

The Hierarchy of EvidenceThe Hierarchy of Evidence

1. Systematic reviews & meta-analyses

2. Randomised controlled trials

3. Cohort studies

4. Case-control studies

5. Cross sectional surveys

6. Case reports

7. Expert opinion

8. Anecdotal

WashingtonMonument

A Year of MEDLINEindexed journals

555 feet

Medical Publishing

Annually:

• 20,000 journals

• 17,000 new books

MEDLINE:

• 4,000 journals

• 6 Million references

• 400,000 new entries yearly

Words used by 41 doctors to describe their information

supply• Impossible Impossible Impossible

Impossible Impossible Impossible• Overwhelming Overwhelming

Overwhelming Overwhelming Overwhelming Overwhelming

• Difficult Difficult Difficult Difficult• Daunting Daunting Daunting• Pissed off• Choked• Depressed• Despairing• Worrisome• Saturation

• Vast• Help• Exhausted• Frustrated• Time consuming• Dreadful• Awesome• Struggle• Mindboggling• Unrealistic• Stress• Challenging Challenging Challenging• Excited• Vital importance

The information paradox

• “Doctors are overwhelmed with information but cannot find information when they need it”

• “Water water everywhere, nor any drop to drink”

My students are dismayed when I say to them “Half of what you are taught as medical students will in 10 years have been shown to be wrong. And the trouble is, none of your teachers knows which half.”

(Dr Sydney Burwell, Dean of Harvard Medical School)

The Slippery Slope

Years since Med School graduation

Knowledgeof best current HTNcare

r = -0.54r = -0.54p<0.001p<0.001. ... ..

. . .... . . . .... . ............ ........

........

......

....

......

Shin,et al: CMAJ;1993: 969-976

Adapted from Slawson et al, J Fam Pract 1994; 38:505-513

Usefulness ofUsefulness ofMedical InformationMedical Information =

RelevanceRelevance x Validity

Work

A Paradigm Shift for Physicians

• From Memory Repositories

• To Information Managers

• From “How do I keep up with new developments in medicine?”

• To “What developments in medicine do I need to keep up with?”

1. Burn your (traditional) textbooks

• For a textbook to be dependable in the modern era:

· it should be revised frequently (at least once a year)

· it should be heavily referenced, at least for declarations about diagnosis and management (so readers can get to original sources for details and can also easily determine the date of a given claim)

UpToDate

Thrombolytic therapy in MI

21

5

101

1 2

8

7

8

12

4

3

1

1

2

8

7

2

1

1

1

2

8

1

5

15

6

No

t M

en

tio

ne

d

Ro

uti

ne

Ex

pe

rim

en

tal

Ra

re/N

ev

er

Sp

ec

ific

M

M

M

M

M

M

Textbook/ReviewRecommendations

Odds Ratio (Log Scale)

0.5 1.0 2.0

Favours Treatment Favours Control

RCTs Pts

1 23

2 65

3 149

4 316

7 1793

10 254411 265115 331117 392922 5452

P<.01

23 5767

27 612530 634633 657143 21 05954 22 051

67 47 53165 47 185

70 48 154

P<.001

P<.00001

CumulativeYear

1965

1970

1980

1985

1990

Antman EM, et al. JAMA, 268:240-8, 1992

Lag time from time of “knowing” to time of implementation

• 13 yrs for thrombolytic therapy.

• 10 yrs for corticosteroids to speed fetal lung maturity.

Our textbooks are out-of-date

• Fail to recommend Rx up to ten years after it’s been shown to be efficacious.

• Continue to recommend therapy up to ten years after it’s been shown to be useless.

2. Invest in evidence databases

• ACP Journal Club http://www.acponline.org/journals/acpjc/jcmenu.htm

• Best Evidence www.acponline.org/

• Cochrane Library

• UpToDate

• MEDLINE

• Harrison’s Online

• Medscape www.medscape.com/Home/Topics/homepages.html

• MD Consult www.mdconsult.com

Does CME Work?

• Davis D A, et al. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-1.

• Sibley J C, A randomized trial of continuing medical education. N Engl J Med 1982; 306: 511-5.

• Conclusion– Traditional CME in a nice place with

pleasant after lecture diversions is, unfortunately, completely ineffective in changing our behavior.

• Another reason to read!

What About Guidelines?

• Guidelines can be very useful– Problems

• Surprise! They don’t all agree.

• Which ones do we use? (Determining validity)

• How do we implement? (How do we remember to do what they say?)

– Once validity is established they can be an excellent resource

The Experts ?

• Remember, they’re in the same position we are with information overload.

• They often look at a patient and a disease in a fundamentally different way because they deal with a selected patient population.

• Excellent resource once reliability has been established.

Evidence Based

Medicine

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

Ask

• We need it twice for every 3 outpatients and 2 times for every inpatient

• Questions are most likely to be about treatment

• Most of the questions generated in consultations go unanswered.

EBM process

When caring for patients creates the need for information:

1 Translation to an answerable question

(patient/maneuver/outcome).

4 Parts of ACQ (PICO)

• P – Patient and problem

• I – Intervention (treatment, test, prognostic factor, etiology, etc.)

• C – Comparison (if necessary)

• O - Outcome

Treatment

• P – In a child with frequent febrile seizures

• I – would anticonvulsant therapy

• C – compared to no treatment

• O – result in seizure reduction?

Diagnosis

• P – In an otherwise healthy 7-year-old boy with sore throat,

• I - how does the clinical exam

• C – compare to throat culture

• O – in diagnosing GAS infection?

Prognosis

• P - In children with Down syndrome,

• I - is IQ an important prognostic factor

• C –

• O – in predicting Alzheimer’s later in life?

Etiology/Harm

• P – controlling for confounding factors, do otherwise healthy children

• I - exposed in utero to cocaine,

• C – compared to children not exposed

• O – have increased incidence of learning disabilities at age six years?

Good ACQ?

• Is Amoxicillin an effective treatment for children with otitis media?

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process

2Efficient track-down of the best evidence –Secondary (pre-appraised) sources e.g.,– Cochrane (systematic reviews)

– E-B Journals

–primary literature

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process

3-Critical appraisal of the evidence for its validity and clinical applicability

• Perhaps most published articles belong in the bin, and should certainly not be used to inform practice.

Why do I have to bother?Can’t I trust the editors?

Percent of articles meeting quality criteria

NEJM 12.6

Ann Int Med 7.6

JAMA 7.2

Lancet 6.2

BMJ 4.4

Arch Int Med 2.4

The Hierarchy of EvidenceThe Hierarchy of Evidence

1. Systematic reviews & meta-analyses

2. Randomised controlled trials

3. Cohort studies

4. Case-control studies

5. Cross sectional surveys

6. Case reports

7. Expert opinion

8. Anecdotal

EBM process

1. Ask

2. Search

3. Appraise

4. Apply

5. Evaluate

EBM process4 Integration of that critical appraisal

with clinical expertise and the patient’s unique biology and beliefs action.

5 Evaluation of one’s performance.

"I know that most men (sic), including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit

the falsity of conclusions which they have delighted in explaining to colleagues, which they

have proudly taught to others, and which they have woven, thread by thread, into the fabric of

their lives.” Leo Tolstoy

"I know that most men (sic), including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit

the falsity of conclusions which they have delighted in explaining to colleagues, which they

have proudly taught to others, and which they have woven, thread by thread, into the fabric of

their lives.” Leo Tolstoy

Recommended Reading

• Gordon Guyatt, Drummond Rennie. Users’ Guides To The Medical Literature, A Manual for Evidence-Based Clinical Practice. AMA.

• Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based Medicine:How to Practice and Teach EBM. Second Edition.Churchill Livingstone: Edinburgh, 2000

• Trisha Greenhalgh : How to read a paper; the basis of evidence based medicine. BMJ 2001.

Thank You!