Clinical Problem Solving an introduction to Evidence-Based physical therapy basics.
-
Upload
colin-gregory-robinson -
Category
Documents
-
view
219 -
download
3
Transcript of Clinical Problem Solving an introduction to Evidence-Based physical therapy basics.
Clinical Problem Solving an introduction to Evidence-Based physical therapy basics
Lecture overview
• Objectives • EBM skills for practicing physical therapy
– Asking– Acquiring– Assessing – Applying
Objectives
• Define evidence-based physical therapy(EBM) • Explain why we use EBM
– Compare with expert-based physical therapy– How are we misled by:
• Surrogate outcomes • Personal observation • Pathophysiologic reasoning
• Describe the tools of EBP• Construct a well-built clinical question
What is Evidence-Based physical therapy(EBM)?
“Using the best available evidence for making decisions about health care”
What is important to read in the medical literature
Things that 1. Have patient oriented outcomes 2. Answer a patient-care question3. Might change your practice4. Are on a topic you have been following5. People are talking about and you want to know more6. You find interesting POEM or DOE
• Patient-oriented evidence that matters vs disease-oriented evidence
“Intro”: EBM (I)
“Evidence-based physical therapy(EBM) requires the integration of the best research evidence with our clinical expertise and our patient’s unique values and circumstances”
EBM, 2006, Straus et al
Why learn EBM / EBP?
“Good education teaches us to become both producers
of knowledge &
discerning consumers of what other people claim to know.”
Helps you find the truth in face of pharmaceutical marketing
Cal Ripkin, Jr. is not hypertensive and is not taking
PRINIVIL
Cal Ripkin, Jr. is not hypertensive and is not taking
PRINIVIL
Its always in the fine print.
Value of Learning EBM:
Short-Term Trial • A controlled trial of teaching critical appraisal of clinical
literature conducted among medical students• Experimental group of students worked with clinical tutors
who had– Taken course in clinical appraisal– Evaluated specialty-specific articles on diagnostic tests and
treatments
• Control group of students worked with usual clinical tutorsBennett et al. JAMA. 1987;257:2451-2454.
Value of Learning EBM: Short-Term Trial (cont)
• Students in experimental group made greater number of correct diagnostic and treatment decisions and were better able to justify their decisions
• Students in control group were more likely to make incorrect decisions after their tutorial than before it– Students in the control group had become more accepting of
recommendations from authority figures
Bennett et al. JAMA. 1987;257:2451-2454.
The Patient
• Patient is a 27-year-old woman with severe right lower quadrant pain. – initial peri-umbilical pain x 2 days migrating yesterday to
current site.
• Loss of appetite. No vomiting, diarrhea; no bowel movement
• no known infectious exposure/
suspicious ingestions, or recent travel
Standard medical practice for hot, moist diseases
Louis’ Study of Bloodletting
Day of 1st bleeding
Averages
Number of bleedings
Duration of illness
Pierre Louis (1787-1872)Inventor of the “numeric method” and the “method of bservation”
Discovered in 1828 that patients who were bled did worse than those who weren’t
Many advances in physical therapywith uncontrolled use
• PCN for life-threatening disease
• Insulin for type I diabetes
• Treatment of malignant hypertension
Traditional Guides to Medical Practice
• Pathophysiology and pharmacology– Foundation of medical practice– Do what “makes sense”
• Expert opinion– In training: learning at the bedside from the master clinician– In practice: lectures and seminars with thought leaders
• Clinical experience– Successes, outcomes, and adverse events
in our own practice
1498 subjects with suppressible arrhythmias post-MI
RANDOMIZED
7.7% 3.0%
Treatment Placebo
Mortality
Cardiac Arrhythmia Suppression Study
Problems With the Traditional Approach• Physiology may not predict clinical response
– Beta-adrenergic blockade in heart failure– Encainide for post-MI arrhythmia– Estrogen replacement for cardioprotection
• Expert opinion– Only as good as the expert– May be affected by biases and conflicts of interest
• Clinical experience– Dramatic clinical experiences may unduly influence our practice
patterns– May not take account of recent medical literature
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-BasedClinical Practice. Chicago, IL: American Medical Association; 2001.
Paradigms of physical therapy
Expert Based Evidence BasedPathophysiological reasoning Clinical Studies
Personal observation Best evidence available
Expert based guidelines Evidence based guidelines
Seven alternatives to EBMHumorous approach from BMJ
Basis Marker Unit
Evidence RCT Odds ratio
Eminence White hair Optical dentistry
Vehemence Level of stridency Decibels
Eloquence Smoothness Adhesion score
Providence Religious fervor International units of piety
Diffidence Level of gloom Sighs
Nervousness Litigation phobia level
Bank Balance
Confidence* Bravado No sweat
BMJ 1999;319:1618-1618
Integrates Evidence With
• Clinical expertise– Experience – Judgment
• Patient values and preferences• Quality of life • Costs • Other important factors
Best Available Evidence
Clinical Expertise
Patient Values and Preferences
Quality of Life
Costs
Focus: Treatment & Diagnosis
The Patient
• Patient is a 27-year-old woman with severe right lower quadrant pain.
– initial peri-umbilical pain x 2 days migrating yesterday to current site.
• Loss of appetite.
• No vomiting, diarrhea; no bowel movement
• no known infectious exposure/ suspicious ingestions, or recent travel
Patient exam
VS BP 120/78 P 16 RR 12 T 98.8
Chest CTA. CV RRR s M/R/G
ABD: NML exam x decreased bowel tones and definite right lower quadrant pain, specifically at McBurney’s point.
no heptomegaly nor splenomegaly (enlarged liver or spleen). She has no rebound pain or involuntary
The Five “A’s”
1. Ask the right question
2. Acquire the evidence
3. Appraise the evidence
4. Apply the evidence
5. Assess its impact
Concern:
• Case discussion: 27 year old woman with right lower quadrant (RLQ) abdominal pain
• Background information available from textbooks-– What typically presents as RLQ pain
– What is the clinical course of the different diagnoses
– Specifically, what is typical presentation of appendicitis
• Foreground information– How good is a CT scan for appendicitis?
Formulating the Question
Formulating the Question• An ideal question:
– Focused enough to be answerable
– Pertinent to clinical scenario
– Framed as Population receiving an Intervention (test or treatment) [as Compared to other test/treatment or placebo] associated with Outcome (disease or improvement)
PICOS
P roblem/population
I ntervention
C omparison
O utcome
S tudy design
Examples of tough questions
• Should I screen men for prostate cancer?
• Who is a good candidate for hormone replacement therapy?
• Are angiotensin receptor blockers now first-line for hypertension?
Examples of better questions• Would a PSA test reduce mortality in a 65 year-old
asymptomatic man? • What is the reduction in fracture risk associated
with hormone replacement therapy? • Is losartan more effective than atenolol at
preventing cardiovascular events in middle-aged hypertensive diabetic women?
PICOS
PICOS for confirmatory diagnosis of appendicitis
P: 27 year old woman with symptoms suggestive of appendicitis
I: CT Scan
C: Ultrasound
O: Accurate diagnosis without undue delay
S: ??
Important Outcomes• Patient Oriented Outcomes:
outcomes patients actually care about – Death (overall or disease-specific)– Heart attacks, strokes, amputations, bed sores, broken hips, renal
failure, etc.– Ability to perform activities of daily living
Versus• Disease oriented outcomes:
– Biochemical, physiologic, pharmacologic, or laboratory measures
Comparing DOE and POE
Shaughnessy AF, Slawson DC. Getting the Most from Review Articles: A Guide for Readers and Writers. American Family Physician 1997 (May 1);55:2155-60.
ExampleDisease-Oriented
Evidence
Patient-Oriented Evidence that
Matters CommentAntiarrhythmic Therapy
Drug X PVCs on ECG
Drug X increases mortality
POE contradicts DOE
Type 2 Diabetes Aggressive Tx with insulin or oral agentscan keep BS low
Aggressive Tx does not reduce mortality or prevent most complications
POE contradicts standard teaching
ProstateScreening
PSA screening detects prostate cancer early
Does PSA screening mortality?
DOE exists, but POE is unknown
Background versus foreground information
• Case discussion: 27 year old woman with right lower quadrant (RLQ) abdominal pain
• Background information available from textbooks-– What typically presents as RLQ pain
– What is the clinical course of the different diagnoses
– Specifically, what is typical presentation of appendicitis
• Foreground information– How good is a CT scan for appendicitis?
Steps of EBM-5 A’s
• Ask• Acquire• Appraise• Apply• Assess
“Finding Evidence”: Sources (I)
• Primary research database (articles)– PubMed (aka MEDLINE), Pyschlit, CCTR
• Secondary research databases (synthesis)– Cochrane Library, Clinical Evidence, InfoPOEMS,
UpToDate
• Tertiary resources (meta search engines, databases of databases)– TRIP+ (Translating Research Into Practice),
PrimeEvidence
“Finding Evidence”: Sources • PubMed
– 16 million peer reviewed biomedical articles indexed (note can use PubMed limits to search on particular populations, study types, etc.)
• Cochrane Library– ~3000 clinical systematic reviews (gold standard database)
• Clinical Evidence– ~2500 tsystematic reviews of treatment classified by likelihood of benefit
• InfoPOEMS (www.infopoems.com)– ~3000 regularly updated entries, Patient Oriented Evidence the Matters
(POEM), 100+ journals monitored• UpToDate
– 70,000 pages, evidence based clinical information resource, ~3000 authors, 350+ journals monitored, peer reviewed
• TRIP+– Meta-search of 55 sites of evidence based information
“Finding Evidence”: Searching1. Convert clinical question to searchable question (e.g. PICOS)
2. Choose the database you want to search (e.g. PubMed)
3. Apply filters to restrict your search (e.g. PubMed limits linked to
PICOS such as gender, age, study type limits)
4. Assess result (e.g. using systematic review worksheet)
5. Decide if you have enough information to make a decision
6. If not then refine steps 1-3 until you either have an answer or decide
there isn’t enough evidence to make an evidence based decision
Appraising the Evidence
Assess the Evidence• Is the study valid?
– Validity is defined as relative freedom from bias and confounding factors• What are the results?
– What is the outcome and how was it measured?– What is the magnitude of the effect?– Are the results statistically significant?
• Do the results apply to my patient?– Does my patient resemble those in the study?– Were all outcomes relevant to my patient evaluated?– Are there other factors (eg, cost, availability) that limit applicability to my
patient?
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001
Objectives
• Understand difference between observational and experimental studies
• For 2 major study designs (randomized controlled trial and cohort study) describe– How the study is designed– Advantages and disadvantages of design– How to assess validity– How to assess results– How to assess applicability
Experimental vs Observational Studies
• In experimental studies, the investigator controls subjects’ exposure to intervention
– Example: randomized controlled trial (RCT)
• In observational studies, investigator does not control the exposure; it occurs naturally or is initiated by patients or their physicians
– Examples: cohort study, case-control study
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
RCTs
Generally held to be the optimal methodology for determining benefit or harm
Eligible Patients
Randomization
Treatment
Control Outcome
Outcome
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
RCTs: Advantages
• Treatment and control groups are likely to have similar distribution of known and unknown prognostic factors (potential confounders)
• Outcomes are determined prospectively in a standardized, systematic fashion
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
RCTs: Disadvantages
• Costly to perform• Size limitations make detection of rare events
difficult (eg, adverse medication effects) • Eligibility restrictions may reduce applicability to
real patients• Cannot be ethically performed if exposure is
expected to cause harm (eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
RCTs: Disadvantages• Costly to perform• Size limitations make detection of rare events
difficult (eg, adverse medication effects) • Eligibility restrictions may reduce applicability to
real patients• Cannot be ethically performed if exposure is
expected to cause harm (eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Assessing the Validity of RCTs• Was randomization concealed?• Were patients analyzed in groups to which they were
randomized?• Were patients in treatment & control groups similar with
respect to prognostic factors?• Were patients, clinicians, outcome assessors, and data
analysts aware of allocation?• Were groups treated equally?• Was follow-up complete?Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice . Chicago, IL: American
Medical Association; 2001.
Assessing the Results of an RCT• Magnitude of result: How large was the treatment effect?
– Relative risk and odds ratio– Absolute risk reduction and number needed to treat (NNT)
• Statistical significance– P value– Confidence interval: How precise was estimate of treatment
effect?• Clinical significance
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Assessing the Results of an RCT• Magnitude of result: How large was the treatment effect?
– Relative risk and odds ratio
– Absolute risk reduction and number needed to treat (NNT)
• Statistical significance– P value
– Confidence interval: How precise was estimate of treatment effect?
• Clinical significance Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago,
IL: American Medical Association; 2001.
Calculating the Risk Ratio and Number Needed to Treat (NNT)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Treatment(n = 1000)
Control(n = 1000)
100 have the outcome
120 have the outcome
Risk = 0.1, or 10%(100/1000)
Risk = 0.12, or 12%(120/1000)
Risk ratio = 0.1/0.12 = 0.83, or 83%
Absolute risk reduction = 0.12 - 0.1 = 0.02, or 2%
NNT = 1/0.02 = 50
Assessing the Applicability of an RCT
• Were the study patients similar to my patient?– Eligibility criteria– “Table 1” data (baseline characteristics)
• Were all clinically important outcomes considered?• Are the likely treatment benefits worth the potential
harm and costs?Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago,
IL: American Medical Association; 2001.
Cohort Studies
• Similar to RCTs, except that assignment to intervention is not random
Eligible Patients
Choice or Happenstance
Exposed
Not Exposed
Outcome
Outcome
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Cohort Studies: Advantages• Outcomes are determined prospectively in a
standardized, systematic fashion• Often includes a larger, more diverse population
than those eligible for or included in RCTs• Can be used to assess effects of harmful exposures
(eg, smoking)
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001
Cohort Studies: Disadvantages
• Costly to perform• Size limitations make detecting rare events
difficult• Exposure and control groups are likely to differ in
factors that may affect outcomes• Control of confounding through statistical analysis
may be inadequateGuyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Assessing the Validity of a Cohort Study
• Were the exposed and control groups similar in all known determinants of outcome? – Did the analysis adjust for potential differences?
• Were the outcomes measured in the same way in the groups being compared?
• Was follow-up sufficiently complete?
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Assessing the Results of a Cohort Study
• How strong is the association between exposure and outcome?– Risk ratio or odds ratio– Absolute risk increase or number needed to harm (NNH)
• Statistical significance– P value– Confidence interval: How precise was estimate of risk?
• Clinical significance
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Assessing the Applicability of a Cohort Study
• Were the study patients similar to the patient under consideration in my practice?
• Should I attempt to stop the exposure?
Guyatt et al. Users' Guides A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001 to the Medical Literature:.
Case-Control Studies
• In contrast to RCTs and cohort studies, participants are selected based on the presence of the outcome rather than the exposure
• Exposure status is determined retrospectively
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Case-Control Studies: Design
Select Subjects: Cases (diseased)
Controls(nondiseased)
Exposed Not ExposedObserve: Exposed Not Exposed
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Case-Control Studies: Advantages
• Much more efficient for investigation of rare outcomes
• Take less time to perform than RCTs or cohort studies
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Case-Control Studies: Disadvantages
• Retrospective assessment of exposure may be inadequate (recall bias)
• Can be performed only after outcomes have occurred (ie, after damage has already occurred)
• Selection of appropriate controls may be difficult• Control of confounding through statistical analysis may
be inadequate
Guyatt et al. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: American Medical Association; 2001.
Steps of EBM-5 A’s
• Ask• Acquire• Appraise• Apply• Assess
Applying EBM
Clinical Expertise
Patient Values and Preferences
Quality of Life
Costs
Best Available Evidence
Integrates Evidence With
• Clinical expertise– Experience – Judgment
• Patient values and preferences• Quality of life • Costs • Other important factors
Integrates Evidence With
• Clinical expertise– Experience – Judgment
• Patient values and preferences• Quality of life • Costs • Other important factors
“Therapy Review” -VALIDITY• Clearly focused question? • Randomization• Blinding- subjects, providers, investigators• Groups similar at start and treated the same throughout?• Followed in randomized groups and accounted for at end? (intention to treat)• Enough subjects to minimize chance differences?
REUSLTS AND PRECISION1. What are results? How presented?2. Certainty & precision? (95% CI’s)
APPLICABILITY1. Can the results be applied to my patient?2. All important outcomes addressed?3. Should there by change in policy?
“Therapy”: Intention to treat
• Subjects are analyzed in the groups they were randomized to.– Maintains randomization– Better reflects real world outcomes– Measures efficacy (“Will this work?”)– Detects issues about intervention other than
effectiveness “In the best possible circumstances, do they work?”
“Therapy”: Bias
• Randomization helps lessen patient bias
– Self-selection
• Blinding helps lessen patient and investigator bias
“Therapy”: What Are the Results ?
• RR OR RRR
• ARR
• NNT / NNH
• P value/ CI
• Clinically significant?
“Therapy”: What Are the Results ?
• RR OR RRR
• ARR
• NNT / NNH
• P value/ CI
• Clinically significant?
“Therapy”: Expressing Results
Risk = outcome event rate
= number having event number receiving the intervention
Relative risk = risk in intervention group(RR) risk in control group
Relative risk reduction (RRR) = 1 - RR
“Therapy”: Expressing Results
Absolute risk reduction (ARR) = difference in risk (control –
intervention)
“Therapy”: Expressing Results
Number-needed-to-treat (NNT) = 1/ARR
in words: the number of patients who need to be treated to prevent one outcome event from occurring in specified time
“Therapy”: Example
• An oncology trial testing a new treatment with 4-year follow-up for mortality
• experimental treatment: 30%• control group: 50%
• What are the RR, RRR, ARR, NNT?
“Therapy”: Example
RR = risk of death in experiment/control groups
= 30%/50% = 0.6 or 60%
RRR = 1 - RR = 1-0.6 = 0.4 or 40%
ARR = risk of death in control – experimental groups
= .50 -.30 = 0.2 or 20%
NNT =1/ARR = 1 ÷ 0.2
= 5 patients treated with the experimental therapy to prevent one death at 4 years
“Therapy”: Relative Versus Absolute Benefits
• Consider – July 3, 2002: Worldcom stock rose 120%
(relative increase)– The stock rose from: $0.10 $0.22 (absolute
increase)
“Therapy”: Relative risk reduction versus absolute risk reduction
• Baseline risk 10/100 5/100 RRR = 50% ARR = 5% NNT = 20
• Baseline risk 1/100 0.5/100RRR = 50% ARR = 0.5% NNT = 200
• Baseline risk 0.1/100 0.05/100RRR = 50% ARR = 0.05% NNT = 2000
“Therapy”: RRR Lipid Trials 4S WOSCOPS CARE AFCAPS
for acute myocardial infarction
RRR 27 31 25 40(%)NNT 19 42 40 435(5 year)
“Therapy”: 95% Confidence Interval• Any statistic only an estimate of the “true value” of that
statistic. • Confidence Interval (CI) gives range within which that “true
value” probably lies. • 95% CI - if we repeated the experiment with similar
populations an infinite number of times, the results would fall within the CI 95% of the time. 95% certain that the “true value” will fall within the 95% CI range.
• CI gives us an idea of the precision of the result, since the narrower the CI is, the more certain we can be that the experimental value is close to the “true value”.
• And, generally, the larger the sample size, the narrower the CI
• CI =idea of significance, e.g.– If the 95% CI for the ARR includes 0, no difference between the
experimental and control groups.
– If the 95% CI for the RRR or Odds Ratio includes 1, – no difference between the experimental and control groups.
• Similar to P values (e.g., P<0.05) =statistically significant
• CI gives a sense of the size of the differences found in the study.
• e.g., research study - 50% of patients treated with Drug A are cured, compared with 45% of patients treated with Drug B.
• ARR I s thus 5%.• Statistical analysis P<0.001, statistically significant.• But if 95% CI of ARR is 0% to 10%, indicates result is not clinically
significant (includes “0%” - “no difference”).
“Therapy”: Statistical Significance Clinical Significance
• Are the results clinically important?
– Duration of pharyngitis: 8.1 days to 7.4 days
– Weight: 279 lbs to 266 lbs after 3 months
– Survival increased from 4.5 mos to 5.2 mos with 100% mortality at
12 months
– Claudication: Increase in walking distance by 34 ft.
“Diagnosis Review” - WorksheetVALIDITY• Clearly focused question? • Appropriate reference standard?• Reference standard & test applied to all subjects? (verification bias)• Did results of standard influence interpreting test results? (review bias)• Disease status reported and varied? (spectrum bias)• Test method reported with sufficient detail to be replicated?
REUSLTS AND PRECISION1. What are results?2. Certainty & precision? (95% CI’s)
APPLICABILITY1. Can the results be applied to my patient?2. Are local resources (equipment, expertise, cost) sufficient to apply these
results?
“Diagnosis”: 2X2 table Diagnostic test characteristics
• Sensitivity
• Specificity
• Predictive Value
• Likelihood Ratios
DIAGNOSTIC TEST
D I S E A S EPresent Absent TOTALS
Test positive
True
Positive
False
Positive
All positive
Test negative
False
Negative
True
Negative
All negative
TOTALSAll with
disease
All without disease
Entire population
T
E
S
T
“Diagnosis”: What are the Results?
(+) (-)
(+)a b
(-)c d
D I S E A S E
T
E
S
T
“Diagnosis”: What are the Results?
Pt has diseaseDz (+)
Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Pt has diseaseDz (+)
Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Dz (+) Pt does not have disease Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Pt has diseaseDz (+)
Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Pt has diseaseDz (+)
Dz (-)
Test(+)a b
Test(-)c d
TP
FN
“Diagnosis”: Sensitivity
Sensitivity is proportion of people with disease who have a positive test
Dz (+) Dz (-)
Test(+)a b
Test(-)c d
Sensitivity = (a/a+c) =(TP/TP+FN)
TP
FN
“Diagnosis”: What are the Results?
Dz (+) Pt does not have disease Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Dz (+) Pt does not have disease Dz (-)
Test(+)a b
Test(-)c d
“Diagnosis”: What are the Results?
Dz (+) Pt does not have disease Dz (-)
Test(+)a b
Test(-)c d
FP
TN
“Diagnosis”: Specificity
Specificity is proportion of people without disease who have negative test
Dz (+) Dz (-)
Test(+)a b
Test(-)c d
Specificity = (d/b+d) =(TN/FP+TN)
FP
TN
“Diagnosis”: Tradeoffs of sensitivity & specificity labeling diabetes
Blood sugar
• 70• 100• 130• 160• 200
Sensitivity Specificity
98.6% 8.8%88.6% 69.8%64.3% 96.9%47.1% 99.8%27.1% 100%
“Diagnosis”: Choosing a test
• SnNout-
A sensitive test, if negative, rules out a disease
• SpPin-
A specific test, if positive, rules in a disease
“Diagnosis”: Sensitivity & Specificity
• Useful for picking a test (test properties)– Screening- prefer sensitive test– Diagnosis – prefer specific test
• Less help in making diagnosis
“Diagnosis”: What are the Results?
• In patients, what you know are their test results- you are trying to determine whether they actually have the disease.
• Positive Predictive Value :
– Of all who tested positive for a disease, the proportion that actually has it
• Negative Predictive Value :
– Of all who tested negative for a disease, the proportion that actually does not have it
“Diagnosis”: What are the Results?
(+) (-)
(+)a b
(-)c d
D I S E A S E
T
E
S
T
“Diagnosis”: Positive Predictive Value
Proportion of people with a positive test who have a disease
PPV = a/a+b=
TP/TP+FP
Dz (+) Dz (-)
Test(+) TPa
FPb
Test(-)c d = true positives
over all positives
“Diagnosis”: Negative Predictive Value
Proportion of people with a negative test who don’t have a disease
NPV= d/d+c=
TN/TN+FN = true negatives
over all positives
Dz (+) Dz (-)
Test(+)a
FPb
Test(-) FNc
TNd
“Diagnosis”: What are the Results?
PPV dependent on prevalence, even when using the same test
• Example: Prevalence of a particular disease in a population is 50%. Sensitivity= 90% Specificity= 95%
PPV: = a/a+b
= 95%
Dz (+) Dz (-)
Test(+)a b
Test(-)c d
100 100 200
90
10 95
5
“Diagnosis”: PPV & prevalence
PPV dependent on prevalence using same test
• Example: Prevalence of a particular disease in a population is 5%. Same test, same sensitivity & specificity– Sensitivity= 90% Specificity= 95%
PPV: = a/a+b
= 47%
Dz (+) Dz (-)
Test(+)a b
Test(-)c d
10 190 200
9
1 180
10
“Diagnosis”: PPV & NPV
• Useful for diagnosis– Probability of disease after (+ ) or (–) test
• Drawbacks:– Sensitive to prevalence of disease– Prevalence of disease in general population may not be
the same as that of patients you see in clinic/ER.– Not all test results can be categorized as “+” or “-”.
For these reasons, some consider PPV & NPV “Old School”.
“Diagnosis”: What are the Results?
Likelihood Ratios
• Likelihood Ratio is how much more likely is it that someone with this finding has the disease, compared to someone who doesn’t. It does NOT vary with prevalence.
Technically, the + LR is how much more likely someone is to get any positive test result if they have disease, compared to someone who doesn’t.
“Diagnosis”: Likelihood ratio
LR = SENSITIVITY 1 - SPECIFICITY
“Diagnosis”: What do all the numbers mean?
The Likelihood Ratio is a diagnostic weight;
It tells you by how much a given diagnostic test result will raise or lower the probability of having the disorder.
Pretest Probability: the chance that the pt has disease, prior to ordering any tests. This is often an estimation based on clinical experience
Post-test Probability: the chance that the pt has disease, given the results of the test
“Diagnosis”: What are the Results?
What do all the numbers mean?
A LR of 1.0 means the post-test probability is exactly the same as the pretest probability.
A LR >1.0 increases the probability of having the disorder.
A LR<1.0 decreases the probability of having the disorder.
“Diagnosis”: What are the Results?
Likelihood ratios >10 or <0.1 generate large changes from pre- to post-test probability and are generally considered significant.
Strong evidence to rule in/rule out a diagnosis.
Likelihood ratios of 5-10 and 0.1-0.2 generate moderate changes in probability.
Moderate evidence to rule in/rule out a diagnosis.
Likelihood ratios of 2-5 and 0.2-0.5 generate small changes.
Minimal evidence to rule in/rule out a diagnosis
Likelihood ratios 0.5-2 usually have little effect
Figure 1a: Likelihood Ratio Nomogram
LRs = Diagnostic Weights
-45%0.1
-30%0.2
-15%0.5Values between 0 and 1 DECREASE probability of disease
01.0
+15%2
+30%5
+45%10Values greater than 1 INCREASE probability of disease
Change in probability
Likelihood Ratio
From Steve McGee, Evidence Based Physical Diagnosis
Table 2a: Likelihood Ratios of Tests for the Diagnosis of Appendicitis
Likelihood ratio
• DVT– Homan’s sign +LR 1.5– Doppler + LR 39
• ANEMIA– Conjunctival rim pallor +LR 16.7
Summary Diagnostic Test
D I S E A S EPresent Absent TOTALS
Positive True
Positive
False
Positive
ALL Positives
Negative False
Negative
True
Negative
ALL Negatives
TOTALSAll with
disease
All without disease
Entire population
T
E
S
T
Sensitivity Specificity
PPV
NPV
Common pitfalls
• Results reported as relative risk – (ex. Migraines, CVA & OC)
• Results that came from recalculating the data after trial was done – (Post-hoc analysis) (ex. Hot study)
• Over-interpreting results• Relying on just one study (ex. Mg for heart dz),
a poor study, or wrong type of study (ex. HRT)• Confusing statistical significance with clinical
significance (ex. Drugs for BPH)• Not looking at CI’s • Not considering who funded study
Steps of EBM-5 A’s
• Ask• Acquire• Appraise• Apply• Assess
Core of EBP
“Supposing is goodbut finding out
is better.”
Mark Twain