Likelihood Ratiosclinepi.cmcvellore.org/.../10/Likelihood-ratios-Dr... · •Sensitivity (Sn) and...
Transcript of Likelihood Ratiosclinepi.cmcvellore.org/.../10/Likelihood-ratios-Dr... · •Sensitivity (Sn) and...
Likelihood RatiosPrathap Tharyan MD, MRCPsych
Adjunct Professor, Clinical Epidemiology Unit
Prof BV Moses Centre for Evidence-Informed Healthcare
& Health Policy
Christian Medical College, Vellore
9 December 2019
DOCTOR, WHAT DO I HAVE? AND WHAT
SHOULD I DO?
Test accuracy alone is insufficient for accurate
diagnosis
• Sensitivity (Sn) and specificity (Sp) describe the accuracy of performance
of a test but:
• There are often trade-offs that one needs to make regarding sensitivity and specificity.
• These are amplified if the threshold / cut off for a positive or a negative test are not
absolute but vary.
• They are two separate measures; clinicians need a measure that combines true and
false results into one
• Sn and Sp are derived from studies on populations that are often different from ones
that a clinician would see.
• Clinicians want to know if their patients have or do not have the target
condition, given the test‟s results.
• A clinician should be able to use a diagnostic test result to make a clinical
decision about whether to investigate further, treat immediately, or tell a
patient to come back later, or that he/she does not have a problem
Consequence of Diagnostic Errors
• False negative errors, i.e., missing disease that is present.
-can result in people foregoing needed treatment for the disease
- the consequence can be as serious as death
Important to reduce false negative rates in tests (A test should be
sensitive to pick up all with disease)
• False positive errors, i.e., falsely indicating disease
- disease-free are subjected to unnecessary work-up procedures or
even treatment.
- negative impact include personal inconvenience and/or unnecessary
stress, anxiety, etc.
Important to reduce false positive rates in tests (Should be specific in
picking up only those with disease)
Overdiagnosis and Overtreatment
• Overdiagnosis is the diagnosis of an
abnormality that bears no
substantial health hazard and no
benefit for patients.
• Mainly due to the use of increasingly
sensitive screening and diagnostic
tests, as well as broadened definitions
of conditions requiring an intervention,
overdiagnosis is a growing but still
largely misunderstood public health
issue..
The main consequence of
overdiagnosis is overtreatment.
Moynihan et al: BMJ2012;344:e3502doi:10.1136/bmj.e350
Drivers of overdiagnosis
• Technological changes detecting ever smaller “abnormalities”
• Commercial and professional vested interests
• Conflicted guideline panels producing expanded disease definitions and writing
guidelines
• Legal incentives that punish underdiagnosis but not overdiagnosis
• Health system incentives favouring more tests and treatments
• Cultural beliefs that more is better; faith in early detection unmodified by its risks
• Confusion between risk and disease
• Physician‟s fear of missing a disease or not meeting their patients‟ expectations
• Lack of access to or understanding about evidence of lack of benefit of
overdiagnosisMoynihan et al: BMJ2012;344:e3502doi:10.1136/bmj.e350
Bulliard and Chiolero Public Health Reviews (2015) 36:8
BALANCING SENSITIVITY AND SPECIFICITY:
PERSPECTIVES ON RELATIVE IMPORTANCE
• Patients may prefer not to miss a cancer diagnosis and hence may not
mind over-investigation
• Health-administrators may prefer not to spend on un-necessary
investigations or may welcome it (depends on who pays for it)
The utility of a diagnostic test
• Thresholds for testing and for treating
Low pre-test
probability
Uncertain High pre-test
probability
Diagnostic tests should increase the post- test probability of an
accurate diagnosis over the pre-test probability of the diagnosis
(prevalence) and help with triggering treatment decisions
Predictive values help in interpretation of test results for the presence
of disease
Disease
(Reference test)
Present Absent
Index
test
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TNTP+FP+
FN+TN
• Positive Predictive
Value (PPV) is the
probability of disease in
a patient with a positive
(abnormal) test result
• TP / TP+FP• Negative Predictive
Value (NPV) is the
probability of not having
the disease when the
result is negative (normal)
• TN / FN +TN
Predictive values are dependant on prevalence
• Prevalence is an important determinant of the interpretation of the result
of a diagnostic test
• When the prevalence of disease in the population tested is relatively
high – the test performs well
• At lower prevalence, the PPV drops to nearly zero, and the test is
virtually useless
• As Sn and Sp fall, the influence of prevalence on PV becomes more
pronounced!
• PPV derived from hospital populations where the prevalence of disease
is high will over-estimate probability if applied to a community setting
LIKELIHOOD RATIOS
Likelihood Ratios
• A useful single measure of accuracy of a diagnostic test is the likelihood
ratio (LR).
• It‟s a ratio of the likelihood of a positive test result being a true positive
rather than a false positive result; or a negative test result being a true
negative test rather than a false negative result
• The LR is equivalent to a relative risk in other epidemiological studies and
is calculated in the same way
• 95% CI for the LR can be calculated as is done for relative risks
Increasing the utility of a diagnostic test result: Likelihood Ratios
• LR is more stable than predictive values (depends on the ratio of Sn
and Sp not prevalence)
• It is possible to calculate LRs for different test results (e.g. for a
positive or a negative test result) and for different thresholds of test
results
• LRs can be estimated for binary (positive or negative), ordinal (more
than two categories) or continuous (number scale) diagnostic test
outcomes.
• However, ordinal and continuous outcomes are often dichotomized using a cut-
off value to help with the decision-making process.
• LR can be used to determine if the application of a diagnostic test
increases the probability of a target disorder compared to the pre-test
probability of the disorder in a given patient
• LRs can be used to combine the results of multiple diagnostic tests
and can be used to increase the post-test probability for a target
Likelihood Ratio for a positive test (LR+)
Disease
(Reference test)
Present Absent
Index
test
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TNTP+FP+
FN+TN
• How much more likely is a positive
test to be found in a person with the
disease than in a person without it?
• The probability of having a true
positive test result rather than a
false positive test result
• LR+ = sensitivity /(1-specificity)
• (TP/TP+FN) / [1- (TN / FP+TN)]
Likelihood Ratio for a negative test (LR-)
Disease
(Reference test)
Present Absent
Index
test
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TNTP+FP+
FN+TN
• How much more likely is a
negative test to be found in a
person without the disease than
in a person with it?
• The probability that the patient
has a true negative test and not
a false negative test result
• LR - = (1-sens)/spec
• [FN / (TP+FN] / [TN / FP+TN]
LRs with more than two test results
• LR for high probability result=(a/x)/(b/y).
• LR is likelihood of a high probability test result when disease is present
divided by likelihood of a high probability test result when no disease is
present.
• LR for intermediate probability result=(c/x)/(d/y).
• LR for low probability result=(e/x)/(f/y).
• * n=a+b+c+d+e+f.Hayden SR, Brown, MD. Likelihood ratio: A powerful tool for incorporating the results of a
diagnostic test into clinical decision making. Ann Emerg Med 1999; 33:575–80.
Likelihood ratios for levels of serum ferritin in Iron Deficiency
Anaemia
Serum Ferritin (mcg/l) LR for Iron Deficiency
Anaemia
>100 0.08
45 to 99 0.54
35 to 44 1.83
25 to 34 2.54
15 to 24 8.83
<15 51.85
Modified from: Guyatt G et al. Laboratory diagnosis of iron deficiency anaemia. J Gen Intern Med. 1992 Mar–Apr;
7(2):145–53.
The Bayesian Approach to using the Likelihood Ratios in
diagnosis
• In a Bayesian approach, one starts with an initial probability estimate
that is based on one‟s knowledge of disease prevalence or from
one‟s previous experiences.
• This initial probability estimate, termed the prior probability , is then
sequentially modified on the basis of each piece of additional
evidence encountered to form new probabilities, termed posterior
probabilities .
• Bayes‟ Theorem is basically a mathematical recognition of context as
an important factor in decision making.
• In other words no diagnostic test is perfect, and because every test
will be wrong sometimes the likelihood that a test is right will depend
heavily upon its context.
Goodman SN. Toward evidence-based medical statistics. II. The Bayes factor. Ann Intern
Med 1999;130:1005-13.
The Bayesian Approach to using the Likelihood Ratios in
diagnosis
• This approach requires an estimate of the probability of a disease
before any test is ordered (i.e. the „pre-test probability‟)
• Bayes‟s theorem of conditional probability states that the pre-test
odds of a hypothesis being true multiplied by the weight of new
evidence (likelihood ratio) generates post-test odds of the hypothesis
being true.
• Conditional probability is the probability of an event occurring given in
the context of some other event or events
• When used for diagnosis of disease, this refers to the odds of having
a certain disease versus not having that disease
Goodman SN. Toward evidence-based medical statistics. II. The Bayes factor. Ann Intern
Med 1999;130:1005-13.
Bayes Theorem
• Bayes Theorem: P(x/A) = [P(A|x)* P(x)]/P(A)
• Where:
• P(x) = the probability of condition × being present. (prior or pre-test
probability)
• P(A) = the probability of A being present. (test result)
• P(x|A) = the probability of condition × being present given the
presence of A (Posterior or post-test probability).
• P(A|x) = the probability of A being present given the presence of
condition x.
• Bayes‟ Theorem states that the pre-test odds of disease
multiplied by the likelihood ratio yields the post-test odds of
disease.
Converting Probabilities to Odds
• Bayes Theorem:
Post-test Odds = Pre-test Odds LR
• Pre-test odds =
Prevalence /(1-prevalence)
• Post-test probability = post-test
odds/(post-test odds +1)
Pre-test Probability is the Prevalence of the
disease
Disease
(Reference test)
Present Absent
Index
test
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TNTP+FP+
FN+TN
• Prevalence or Pre-test
Probability can be
estimated from prior
knowledge or local data or
calculated from results of a
study
• Pre-test Probability =
TP+FN / TP+FP+FN+TN
Exercise: Serum Ferritin in diagnosis of Iron Deficiency Anaemia
Iron deficiency anemia
(Bone Marrow Iron)Totals
Present Absent
Diagnostic
test result
(serum
ferritin)
Positive
(< 65 mmol/L)
731
TP
a
270
FP
b
1001
a+b
Negative
( 65 mmol/L)
78
FN
c
1500
TN
d
1578
c+d
Totals
809
a+c
1770
b+d
2579
a+b+c+d
What is the post test
probability of Iron deficiency
anemia in a 56 year old man
with a serum ferritin level of
35 who is afebrile and has
pallor?
Exercise: Serum Ferritin in diagnosis of anaemia
Iron deficiency anemia
(Bone Marrow Iron)Totals
Present Absent
Diagnostic
test result
(serum
ferritin)
Positive
(< 65 mmol/L)731
TP
a
270
FP
b
1001
a+b
Negative
( 65 mmol/L)78
FN
c
1500
TN
d
1578
c+d
Totals809
a+c
1770
b+d
2579
a+b+c+d
• Sensitivity = a/(a+c)
• Specificity = d/(b+d)
• Positive Predictive Value =
a/(a+b)
• Negative Predictive Value =
d/(c+d)
• Likelihood ratio for a positive
test result = LR+ = sens/(1-
spec)
• Likelihood ratio for a negative
test result = LR - = (1-
sens)/spec
• Pre-test probability
(prevalence) = (a+c)/(a+b+c+d)
• Pre-test odds = prevalence/(1-
prevalence)
• Post-test odds = pre-test odds
LR
• Post-test probability = post-test
odds/(post-test odds +1)
Exercise: Serum Ferritin in diagnosis of anaemia
Iron deficiency anemia
(Bone Marrow Iron)Totals
Present Absent
Diagnosti
c test
result
(serum
ferritin)
Positive
(< 65
mmol/L)
731
TP
a
270
FP
b
1001
a+b
Negative
( 65
mmol/L)
78
FN
c
1500
TN
d
1578
c+d
Totals
809
a+c
1770
b+d
2579
a+b+c+d
• Sensitivity = a/(a+c) = 731/809 = 90%
• Specificity = d/(b+d) = 1500/1770 =
85%
• Positive Predictive Value = a/(a+b) =
731/1001 = 73%
• Negative Predictive Value = d/(c+d) =
1500/1578 = 95%
• Likelihood ratio for a positive test
result = LR+ = sens/(1-spec) =
90%/15% = 6
• Likelihood ratio for a negative test
result = LR - = (1-sens)/spec =
10%/85% = 0.12
• Pre-test probability (prevalence) =
(a+c)/(a+b+c+d) = 809/2579 = 32%
• Pre-test odds = prevalence/(1-
prevalence) = 0.31/0.69 = 0.45
• Post-test odds = pre-test odds LR =
The Fagan's nomogram.
Charles G B Caraguel, and Raphaël Vanderstichel Evid
Based Med 2013;18:125-128
©2013 by BMJ Publishing Group Ltd
MRI screening test for breast
cancer in high-risk female patients:
Sensitivity = 75%
Specificity = 96%
LR+ (0.75/(1-0.96) = 0.75/0.04
=18.75
LR- (1-0.75)/0.96) = 0.26
A patient from a high-risk population
has an estimated pre-test
probability of 2%
If MR +: post-test probability that
she truly has cancer = ~28% (red
line).
If MR-: post-test probability that she
truly has cancer = ~0.6% (blue line).Warner E, Messersmith H, Causer P, et
al. Systematic review: using magnetic
resonance imaging to screen women at
high risk for breast cancer. Ann Intern
Med 2008;148:671–9
Pre-test probability = 32%
LR+ = 6
Post-test probability test += 73%
LR- = 0.12
Post-test probability test- = 5%
Serum Ferritin in
diagnosis of Iron
Deficiency
Anaemia
Dynamic Fagan Nomogram:
https://cscheid.net/projects/fagan_nomogram/
Two-Step Fagan Nomogram
Charles G B Caraguel, and Raphaël Vanderstichel Evid Based
Med 2013;18:125-128
Charles G B Caraguel, and Raphaël
Vanderstichel Evid Based Med
2013;18:125-128
Two-Step Fagan
Nomogram
• MRI screening test for breast
cancer in high-risk female patients
• Sensitivity of 75% and specificity
of 96%.
• A positive result from the MRI
provides a likelihood ratio (LR+) of
~ 19 (red line, I).
• A patient from a high-risk
population has an estimated pre-
test probability of 2%
• If she tested positive, the post-test
probability for this patient to truly
have cancer would be ~28% (red
line, II).
• A negative test result would
produce a likelihood ratio (LR−) of
approximately 0.25 (blue line, I)
• The post-test probability for this
patient to truly have cancer would
be approximately 0.6% (blue line,
II
Likelihood ratio interpretation at the bedside
McGee, S. Simplifying Likelihood Ratios. J Gen Intern Med. 2002 Aug; 17(8):
647– 650.
Benchmarking LRs and probability of change in post-test
probability
• Remember 3 specific LRs: 2, 5, 10
• And the first 3 multiples of 15 (15, 30,
45)
• LR 2 increases post test probability by
15%, LR 5 by 30% and LR 10 by 45%
• For LRs between 0 and 1, invert 2, 5,
and 10 (i.e.: ½ = 0.5. 1/5 = 0.2 and
1/10 = 0.1).
• Inverse of LR 2 (0.5) decreases
probability 15%,
• Inverse of LR 5 (0.2) decreases
probability 30%
• Inverse of LR 10 (0.1) decreases
probability 45%
• These benchmark LRs can be used to
deduce the restMcGee, S. Simplifying Likelihood Ratios. J Gen Intern Med. 2002 Aug; 17(8):
647– 650.
Benchmarking LRs and probability of change in post-test
probability
• Change in probability ~ 0.19 x log LR
• Regardless of a patient‟s pre-test
probability, the change in probability
from a finding is approximated by a
constant (0.19 x log LR).
• The bedside estimates are rounded off
to the nearest 5% for easy recall
• Not accurate for pre-test probabilities
of less than 10% or greater than 90%
but these do not warrant further tests
• Useful when pre-test probability is not
readily known
McGee, S. Simplifying Likelihood Ratios. J Gen Intern
Med. 2002 Aug; 17(8): 647– 650.
Uses of LR
McGee, S.
Simplifying
Likelihood Ratios. J
Gen Intern Med.
2002 Aug; 17(8):
647– 650.
Using LRs
•
Using LRs in sequence
• When one test increases the post-test probability somewhat and
another test is done, the pre-test probability is now the post test
probability after the first test.
• In clinical practice, the history and physical exam serve to increase or
decrease post-test probability and supplemented by additional tests aid
more accurate diagnoses by acting synergistically,
• For example: While numerous elements of the clinical examination are
associated with the diagnosis of COPD, only 3 are significant on
multivariate analysis. Patients having all 3 of these findings have an LR
of 33 (ruling in COPD); those with none have an LR of 0.18 (ruling out
COPD) [Strauss et al, J Gen Intern Med 2002; 17 (9):684-8]
Limitations of Likelihood Ratios
• The accuracy of a LR depends entirely upon the relevance and
quality of the studies that generated the numbers (sensitivity and
specificity) that inform that LR.
• Clinical decision making occurs by absorbing multiple factors and
generating impressions simultaneously. LRs demand that we
consider one element of diagnosis at a time.
• Some clinicians use one LR to generate a post-test probability, and
then use the new post-test probability as a pre-test probability for
application of the next LR related to a different test. There is no
evidence to support or refute the use of LRs in this fashion
Decision process in making a diagnosis
Lancet 2005; 365: 1500–05
Diagnostic odds ratios can be derived from
LRs
FNFP
TNTPORDiagnostic
veLR
veLR
yspecificit
yspecificit
ysensitivit
ysensitivit
DOR
1
1
Ratio of the odds of positivity in the diseased to the odds of positivity in the non-diseased
Also a stable measure and not affected by prevalence
Disease
(Reference test)
Present Absent
Index
test
+ TP FP TP+FP
- FN TN FN+TN
TP+FN FP+TNTP+FP+
FN+TN
LR CALCULATORS
http://getthediagnosis.org/calculator.htm
http://araw.mede.uic.edu/cgi-bin/testcalc.pl
http://araw.mede.uic.edu/cgi-bin/testcalc.pl?DT=&Dt=&dT=&dt=&2x2=Compute
https://www.medcalc.net/statisticaltests/diagnostic_test.php