Critically Appraising a Harm Study- Part 1

Terry Shaneyfelt, MD, MPH

Associate Professor of Medicine

UAB School of Medicine

ebmteacher.com

Appraising harm studies: 3 steps

1. Are the results valid?

2. What are the results?

3. Will they help me

care for my patients?

1. In a cohort study were there clearly defined groups of patients similar in all important ways other than the exposure of interest?

2. In a case-control study did the cases and controls have a similar chance of being exposed in the past to the factor of interest?

3. Were exposures and outcomes measured the same way in both groups?

4. Was follow-up sufficiently long and complete?

Are The Results Valid?

• Patients in each cohort should be similar for prognostic factors that are known to be assocwith the outcome

• Look at Table 1

• If imbalances are seen….

1. Are they important?

2. Are they dealt with in some way?

3. What else is missing?

1) In a cohort study were the groups similar at the start of the study except for exposure?

Watch my other videos on…

• 3 questions:

1. Are the differences important?– Yes!

2. Were they adjusted for?– Yes

3. Residual confounding? – Likely

1) Were the NHS cohorts

similar?

• Both cases and control should have equal opportunity of being exposed

• Controls:• Should meet same inclusion/exclusion criteria

• Should be at risk for developing the outcome (could become a case)

• What about this study?– Is HRT associated with uterine cancer?

– Cases: women with uterine cancer

– Controls: men without uterine cancer

In a case-control study did the cases and controls have a similar chance of being exposed in the

past to the factor of interest?

Please watch Part 2 of Critically Appraising a Therapy Study

Critically Appraising a Harm Study- Part 2

Terry Shaneyfelt, MD, MPH

Associate Professor of Medicine

UAB School of Medicine

ebmteacher.com

1. In a cohort study were there clearly defined groups of patients similar in all important ways other than the exposure of interest?

2. In a case-control study did the cases and controls have a similar chance of being exposed in the past to the factor of interest?

3. Were exposures and outcomes measured the same way in both groups?

4. Was follow-up sufficiently long and complete?

Are The Results Valid?

• Investigators report a 3-fold increase in risk of melanoma in individuals working with radioactive materials

– What if physicians, concerned about a possible risk, searched more diligently in those exposed to radiation than those who weren't?

• Researchers should:

– Be blinded to exposure status (cohort studies) and to outcome status and hypothesis (case-control studies)

– Use same methods in both groups (standardized)

3) Were exposures and outcomes measured the same way in both groups?

Sources of Exposure Information

• Pre-existing Records• Inexpensive, available,

unbiased

• May be incomplete

• Interviews & Questionnaires (Patients, MDs, Family)

• Complete data

• Bias (e.g. recall bias)

• Proxy Measures (e.g. Job title, Proximity)

• Inaccurate

• Direct Measurement

• Current, future

3) Were exposures and outcomes

measured the same way in both groups?

• Follow-up needs to be long enough for participants to develop the outcome(s) of interest

• Ideally no patients are lost to follow-up

• Loss to f/u can introduce bias if – Those who are lost have different outcomes

– Differences in f/u between exposed and unexposed cohorts

4) Was follow-up sufficiently long and complete?

4) Was follow-up sufficiently long and complete?

• Moderate risk for bias (residual confounding)- in hindsight!

• Options:

– Read the results and realize the findings might not be accurate

– Find another study

–

Assessment of the NHS