Likelihood RatiosPrathap Tharyan MD, MRCPsych

Adjunct Professor, Clinical Epidemiology Unit

Prof BV Moses Centre for Evidence-Informed Healthcare

& Health Policy

Christian Medical College, Vellore

9 December 2019

DOCTOR, WHAT DO I HAVE? AND WHAT

SHOULD I DO?

Test accuracy alone is insufficient for accurate

diagnosis

• Sensitivity (Sn) and specificity (Sp) describe the accuracy of performance

of a test but:

• There are often trade-offs that one needs to make regarding sensitivity and specificity.

• These are amplified if the threshold / cut off for a positive or a negative test are not

absolute but vary.

• They are two separate measures; clinicians need a measure that combines true and

false results into one

• Sn and Sp are derived from studies on populations that are often different from ones

that a clinician would see.

• Clinicians want to know if their patients have or do not have the target

condition, given the test‟s results.

• A clinician should be able to use a diagnostic test result to make a clinical

decision about whether to investigate further, treat immediately, or tell a

patient to come back later, or that he/she does not have a problem

Consequence of Diagnostic Errors

• False negative errors, i.e., missing disease that is present.

-can result in people foregoing needed treatment for the disease

- the consequence can be as serious as death

Important to reduce false negative rates in tests (A test should be

sensitive to pick up all with disease)

• False positive errors, i.e., falsely indicating disease

- disease-free are subjected to unnecessary work-up procedures or

even treatment.

- negative impact include personal inconvenience and/or unnecessary

stress, anxiety, etc.

Important to reduce false positive rates in tests (Should be specific in

picking up only those with disease)

Overdiagnosis and Overtreatment

• Overdiagnosis is the diagnosis of an

abnormality that bears no

substantial health hazard and no

benefit for patients.

• Mainly due to the use of increasingly

sensitive screening and diagnostic

tests, as well as broadened definitions

of conditions requiring an intervention,

overdiagnosis is a growing but still

largely misunderstood public health

issue..

The main consequence of

overdiagnosis is overtreatment.

Moynihan et al: BMJ2012;344:e3502doi:10.1136/bmj.e350

Drivers of overdiagnosis

• Technological changes detecting ever smaller “abnormalities”

• Commercial and professional vested interests

• Conflicted guideline panels producing expanded disease definitions and writing

guidelines

• Legal incentives that punish underdiagnosis but not overdiagnosis

• Health system incentives favouring more tests and treatments

• Cultural beliefs that more is better; faith in early detection unmodified by its risks

• Confusion between risk and disease

• Physician‟s fear of missing a disease or not meeting their patients‟ expectations

• Lack of access to or understanding about evidence of lack of benefit of

overdiagnosisMoynihan et al: BMJ2012;344:e3502doi:10.1136/bmj.e350

Bulliard and Chiolero Public Health Reviews (2015) 36:8

BALANCING SENSITIVITY AND SPECIFICITY:

PERSPECTIVES ON RELATIVE IMPORTANCE

• Patients may prefer not to miss a cancer diagnosis and hence may not

mind over-investigation

• Health-administrators may prefer not to spend on un-necessary

investigations or may welcome it (depends on who pays for it)

The utility of a diagnostic test

• Thresholds for testing and for treating

Low pre-test

probability

Uncertain High pre-test

probability

Diagnostic tests should increase the post- test probability of an

accurate diagnosis over the pre-test probability of the diagnosis

(prevalence) and help with triggering treatment decisions

Predictive values help in interpretation of test results for the presence

of disease

Disease

(Reference test)

Present Absent

Index

test

+ TP FP TP+FP

- FN TN FN+TN

TP+FN FP+TNTP+FP+

FN+TN

• Positive Predictive

Value (PPV) is the

probability of disease in

a patient with a positive

(abnormal) test result

• TP / TP+FP• Negative Predictive

Value (NPV) is the

probability of not having

the disease when the

result is negative (normal)

• TN / FN +TN

Predictive values are dependant on prevalence

• Prevalence is an important determinant of the interpretation of the result

of a diagnostic test

• When the prevalence of disease in the population tested is relatively

high – the test performs well

• At lower prevalence, the PPV drops to nearly zero, and the test is

virtually useless

• As Sn and Sp fall, the influence of prevalence on PV becomes more

pronounced!

• PPV derived from hospital populations where the prevalence of disease

is high will over-estimate probability if applied to a community setting

LIKELIHOOD RATIOS

Likelihood Ratios

• A useful single measure of accuracy of a diagnostic test is the likelihood

ratio (LR).

• It‟s a ratio of the likelihood of a positive test result being a true positive

rather than a false positive result; or a negative test result being a true

negative test rather than a false negative result

• The LR is equivalent to a relative risk in other epidemiological studies and

is calculated in the same way

• 95% CI for the LR can be calculated as is done for relative risks

Increasing the utility of a diagnostic test result: Likelihood Ratios

• LR is more stable than predictive values (depends on the ratio of Sn

and Sp not prevalence)

• It is possible to calculate LRs for different test results (e.g. for a

positive or a negative test result) and for different thresholds of test

results

• LRs can be estimated for binary (positive or negative), ordinal (more

than two categories) or continuous (number scale) diagnostic test

outcomes.

• However, ordinal and continuous outcomes are often dichotomized using a cut-

off value to help with the decision-making process.

• LR can be used to determine if the application of a diagnostic test

increases the probability of a target disorder compared to the pre-test

probability of the disorder in a given patient

• LRs can be used to combine the results of multiple diagnostic tests

and can be used to increase the post-test probability for a target

Likelihood Ratio for a positive test (LR+)

Disease

(Reference test)

Present Absent

Index

test

+ TP FP TP+FP

- FN TN FN+TN

TP+FN FP+TNTP+FP+

FN+TN

• How much more likely is a positive

test to be found in a person with the

disease than in a person without it?

• The probability of having a true

positive test result rather than a

false positive test result

• LR+ = sensitivity /(1-specificity)

• (TP/TP+FN) / [1- (TN / FP+TN)]

Likelihood Ratio for a negative test (LR-)

Disease

(Reference test)

Present Absent

Index

test

+ TP FP TP+FP

- FN TN FN+TN

TP+FN FP+TNTP+FP+

FN+TN

• How much more likely is a

negative test to be found in a

person without the disease than

in a person with it?

• The probability that the patient

has a true negative test and not

a false negative test result

• LR - = (1-sens)/spec

• [FN / (TP+FN] / [TN / FP+TN]

LRs with more than two test results

• LR for high probability result=(a/x)/(b/y).

• LR is likelihood of a high probability test result when disease is present

divided by likelihood of a high probability test result when no disease is

present.

• LR for intermediate probability result=(c/x)/(d/y).

• LR for low probability result=(e/x)/(f/y).

• * n=a+b+c+d+e+f.Hayden SR, Brown, MD. Likelihood ratio: A powerful tool for incorporating the results of a

diagnostic test into clinical decision making. Ann Emerg Med 1999; 33:575–80.

Likelihood ratios for levels of serum ferritin in Iron Deficiency

Anaemia

Serum Ferritin (mcg/l) LR for Iron Deficiency

Anaemia

>100 0.08

45 to 99 0.54

35 to 44 1.83

25 to 34 2.54

15 to 24 8.83

<15 51.85

Modified from: Guyatt G et al. Laboratory diagnosis of iron deficiency anaemia. J Gen Intern Med. 1992 Mar–Apr;

7(2):145–53.

The Bayesian Approach to using the Likelihood Ratios in

diagnosis

• In a Bayesian approach, one starts with an initial probability estimate

that is based on one‟s knowledge of disease prevalence or from

one‟s previous experiences.

• This initial probability estimate, termed the prior probability , is then

sequentially modified on the basis of each piece of additional

evidence encountered to form new probabilities, termed posterior

probabilities .

• Bayes‟ Theorem is basically a mathematical recognition of context as

an important factor in decision making.

• In other words no diagnostic test is perfect, and because every test

will be wrong sometimes the likelihood that a test is right will depend

heavily upon its context.

Goodman SN. Toward evidence-based medical statistics. II. The Bayes factor. Ann Intern

Med 1999;130:1005-13.

The Bayesian Approach to using the Likelihood Ratios in

diagnosis

• This approach requires an estimate of the probability of a disease

before any test is ordered (i.e. the „pre-test probability‟)

• Bayes‟s theorem of conditional probability states that the pre-test

odds of a hypothesis being true multiplied by the weight of new

evidence (likelihood ratio) generates post-test odds of the hypothesis

being true.

• Conditional probability is the probability of an event occurring given in

the context of some other event or events

• When used for diagnosis of disease, this refers to the odds of having

a certain disease versus not having that disease

Goodman SN. Toward evidence-based medical statistics. II. The Bayes factor. Ann Intern

Med 1999;130:1005-13.

Bayes Theorem

• Bayes Theorem: P(x/A) = [P(A|x)* P(x)]/P(A)

• Where:

• P(x) = the probability of condition × being present. (prior or pre-test

probability)

• P(A) = the probability of A being present. (test result)

• P(x|A) = the probability of condition × being present given the

presence of A (Posterior or post-test probability).

• P(A|x) = the probability of A being present given the presence of

condition x.

• Bayes‟ Theorem states that the pre-test odds of disease

multiplied by the likelihood ratio yields the post-test odds of

disease.

Converting Probabilities to Odds

• Bayes Theorem:

Post-test Odds = Pre-test Odds LR

• Pre-test odds =

Prevalence /(1-prevalence)

• Post-test probability = post-test

odds/(post-test odds +1)

Pre-test Probability is the Prevalence of the

disease

Disease

(Reference test)

Present Absent

Index

test

+ TP FP TP+FP

- FN TN FN+TN

TP+FN FP+TNTP+FP+

FN+TN

• Prevalence or Pre-test

Probability can be

estimated from prior

knowledge or local data or

calculated from results of a

study

• Pre-test Probability =

TP+FN / TP+FP+FN+TN

Exercise: Serum Ferritin in diagnosis of Iron Deficiency Anaemia

Iron deficiency anemia

(Bone Marrow Iron)Totals

Present Absent

Diagnostic

test result

(serum

ferritin)

Positive

(< 65 mmol/L)

731

TP

a

270

FP

b

1001

a+b

Negative

( 65 mmol/L)

78

FN

c

1500

TN

d

1578

c+d

Totals

809

a+c

1770

b+d

2579

a+b+c+d

What is the post test

probability of Iron deficiency

anemia in a 56 year old man

with a serum ferritin level of

35 who is afebrile and has

pallor?

Exercise: Serum Ferritin in diagnosis of anaemia

Iron deficiency anemia

(Bone Marrow Iron)Totals

Present Absent

Diagnostic

test result

(serum

ferritin)

Positive

(< 65 mmol/L)731

TP

a

270

FP

b

1001

a+b

Negative

( 65 mmol/L)78

FN

c

1500

TN

d

1578

c+d

Totals809

a+c

1770

b+d

2579

a+b+c+d

• Sensitivity = a/(a+c)

• Specificity = d/(b+d)

• Positive Predictive Value =

a/(a+b)

• Negative Predictive Value =

d/(c+d)

• Likelihood ratio for a positive

test result = LR+ = sens/(1-

spec)

• Likelihood ratio for a negative

test result = LR - = (1-

sens)/spec

• Pre-test probability

(prevalence) = (a+c)/(a+b+c+d)

• Pre-test odds = prevalence/(1-

prevalence)

• Post-test odds = pre-test odds

LR

• Post-test probability = post-test

odds/(post-test odds +1)

Exercise: Serum Ferritin in diagnosis of anaemia

Iron deficiency anemia

(Bone Marrow Iron)Totals

Present Absent

Diagnosti

c test

result

(serum

ferritin)

Positive

(< 65

mmol/L)

731

TP

a

270

FP

b

1001

a+b

Negative

( 65

mmol/L)

78

FN

c

1500

TN

d

1578

c+d

Totals

809

a+c

1770

b+d

2579

a+b+c+d

• Sensitivity = a/(a+c) = 731/809 = 90%

• Specificity = d/(b+d) = 1500/1770 =

85%

• Positive Predictive Value = a/(a+b) =

731/1001 = 73%

• Negative Predictive Value = d/(c+d) =

1500/1578 = 95%

• Likelihood ratio for a positive test

result = LR+ = sens/(1-spec) =

90%/15% = 6

• Likelihood ratio for a negative test

result = LR - = (1-sens)/spec =

10%/85% = 0.12

• Pre-test probability (prevalence) =

(a+c)/(a+b+c+d) = 809/2579 = 32%

• Pre-test odds = prevalence/(1-

prevalence) = 0.31/0.69 = 0.45

• Post-test odds = pre-test odds LR =

The Fagan's nomogram.

Charles G B Caraguel, and Raphaël Vanderstichel Evid

Based Med 2013;18:125-128

©2013 by BMJ Publishing Group Ltd

MRI screening test for breast

cancer in high-risk female patients:

Sensitivity = 75%

Specificity = 96%

LR+ (0.75/(1-0.96) = 0.75/0.04

=18.75

LR- (1-0.75)/0.96) = 0.26

A patient from a high-risk population

has an estimated pre-test

probability of 2%

If MR +: post-test probability that

she truly has cancer = ~28% (red

line).

If MR-: post-test probability that she

truly has cancer = ~0.6% (blue line).Warner E, Messersmith H, Causer P, et

al. Systematic review: using magnetic

resonance imaging to screen women at

high risk for breast cancer. Ann Intern

Med 2008;148:671–9

Pre-test probability = 32%

LR+ = 6

Post-test probability test += 73%

LR- = 0.12

Post-test probability test- = 5%

Serum Ferritin in

diagnosis of Iron

Deficiency

Anaemia

Dynamic Fagan Nomogram:

https://cscheid.net/projects/fagan_nomogram/

Two-Step Fagan Nomogram

Charles G B Caraguel, and Raphaël Vanderstichel Evid Based

Med 2013;18:125-128

Charles G B Caraguel, and Raphaël

Vanderstichel Evid Based Med

2013;18:125-128

Two-Step Fagan

Nomogram

• MRI screening test for breast

cancer in high-risk female patients

• Sensitivity of 75% and specificity

of 96%.

• A positive result from the MRI

provides a likelihood ratio (LR+) of

~ 19 (red line, I).

• A patient from a high-risk

population has an estimated pre-

test probability of 2%

• If she tested positive, the post-test

probability for this patient to truly

have cancer would be ~28% (red

line, II).

• A negative test result would

produce a likelihood ratio (LR−) of

approximately 0.25 (blue line, I)

• The post-test probability for this

patient to truly have cancer would

be approximately 0.6% (blue line,

II

Likelihood ratio interpretation at the bedside

McGee, S. Simplifying Likelihood Ratios. J Gen Intern Med. 2002 Aug; 17(8):

647– 650.

Benchmarking LRs and probability of change in post-test

probability

• Remember 3 specific LRs: 2, 5, 10

• And the first 3 multiples of 15 (15, 30,

45)

• LR 2 increases post test probability by

15%, LR 5 by 30% and LR 10 by 45%

• For LRs between 0 and 1, invert 2, 5,

and 10 (i.e.: ½ = 0.5. 1/5 = 0.2 and

1/10 = 0.1).

• Inverse of LR 2 (0.5) decreases

probability 15%,

• Inverse of LR 5 (0.2) decreases

probability 30%

• Inverse of LR 10 (0.1) decreases

probability 45%

• These benchmark LRs can be used to

deduce the restMcGee, S. Simplifying Likelihood Ratios. J Gen Intern Med. 2002 Aug; 17(8):

647– 650.

Benchmarking LRs and probability of change in post-test

probability

• Change in probability ~ 0.19 x log LR

• Regardless of a patient‟s pre-test

probability, the change in probability

from a finding is approximated by a

constant (0.19 x log LR).

• The bedside estimates are rounded off

to the nearest 5% for easy recall

• Not accurate for pre-test probabilities

of less than 10% or greater than 90%

but these do not warrant further tests

• Useful when pre-test probability is not

readily known

McGee, S. Simplifying Likelihood Ratios. J Gen Intern

Med. 2002 Aug; 17(8): 647– 650.

Uses of LR

McGee, S.

Simplifying

Likelihood Ratios. J

Gen Intern Med.

2002 Aug; 17(8):

647– 650.

Using LRs

•

Using LRs in sequence

• When one test increases the post-test probability somewhat and

another test is done, the pre-test probability is now the post test

probability after the first test.

• In clinical practice, the history and physical exam serve to increase or

decrease post-test probability and supplemented by additional tests aid

more accurate diagnoses by acting synergistically,

• For example: While numerous elements of the clinical examination are

associated with the diagnosis of COPD, only 3 are significant on

multivariate analysis. Patients having all 3 of these findings have an LR

of 33 (ruling in COPD); those with none have an LR of 0.18 (ruling out

COPD) [Strauss et al, J Gen Intern Med 2002; 17 (9):684-8]

Limitations of Likelihood Ratios

• The accuracy of a LR depends entirely upon the relevance and

quality of the studies that generated the numbers (sensitivity and

specificity) that inform that LR.

• Clinical decision making occurs by absorbing multiple factors and

generating impressions simultaneously. LRs demand that we

consider one element of diagnosis at a time.

• Some clinicians use one LR to generate a post-test probability, and

then use the new post-test probability as a pre-test probability for

application of the next LR related to a different test. There is no

evidence to support or refute the use of LRs in this fashion

Decision process in making a diagnosis

Lancet 2005; 365: 1500–05

Diagnostic odds ratios can be derived from

LRs

FNFP

TNTPORDiagnostic

veLR

veLR

yspecificit

yspecificit

ysensitivit

ysensitivit

DOR

1

1

Ratio of the odds of positivity in the diseased to the odds of positivity in the non-diseased

Also a stable measure and not affected by prevalence

Disease

(Reference test)

Present Absent

Index

test

+ TP FP TP+FP

- FN TN FN+TN

TP+FN FP+TNTP+FP+

FN+TN

LR CALCULATORS

http://getthediagnosis.org/calculator.htm

http://araw.mede.uic.edu/cgi-bin/testcalc.pl

http://araw.mede.uic.edu/cgi-bin/testcalc.pl?DT=&Dt=&dT=&dt=&2x2=Compute

https://www.medcalc.net/statisticaltests/diagnostic_test.php

THANK YOU

prathaptharyan@gmail.com

48