TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.
-
Upload
jeffery-surgener -
Category
Documents
-
view
298 -
download
7
Transcript of TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.
![Page 1: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/1.jpg)
TEACHING ABOUT DIAGNOSIS
Tom Sensky
Teaching EBM/EBMH
St Hugh’s College OxfordSeptember 2005
![Page 2: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/2.jpg)
BY THE END OF THIS SESSION,YOU SHOULD BE ABLE TO ….
• describe and illustrate key measures of diagnostic test performance
• describe some less commonly quoted measures of diagnostic test performance
• represent diagnostic test performance in at least four different ways (five if time allows!)
![Page 3: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/3.jpg)
METHOD 1: NATURAL FREQUENCIES GRID
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Assume that the prevalence of the disease is 4%
![Page 4: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/4.jpg)
Assume that of the 4 people with the disease, 3 are picked up by the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
![Page 5: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/5.jpg)
Assume that of the test is positive for a further 7 people who don’t have the disease
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
![Page 6: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/6.jpg)
The remainder of the sample are negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
![Page 7: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/7.jpg)
SENSITIVITY
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
•SENSITIVITY is the proportion of people with the disease correctly identified by the test
•It measures the proportion of false NEGATIVES
![Page 8: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/8.jpg)
SENSITIVITY
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
In this case, sensitivity is ¾ or 75%
![Page 9: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/9.jpg)
SPECIFICITY
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
•SPECIFICITY is the proportion of people without the disease correctly identified by the test
•It measures the proportion of false POSITIVES
![Page 10: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/10.jpg)
SPECIFICITY
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
In this case, specificity is (96-7)/96 or 93%
![Page 11: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/11.jpg)
If someone is positive on the test, what are the chances that he/she has the disease?
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
•Probability = 3/10 = 30%•This is the POSITIVE PREDICTIVE VALUE (the value of the test in predicting a positive result)
![Page 12: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/12.jpg)
If someone is negative on the test, what are the chances that he/she does not have the disease?
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
Person without the disease
Person with the disease
Person who tests positive
Person who tests negative
True positive on the test
False positive on the test
True negative on the test
False negative on the test
•Probability = 89/90 = 99%•This is the NEGATIVE PREDICTIVE VALUE (the value of the test in predicting a negative result)
![Page 13: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/13.jpg)
SENSITIVITY, SPECIFICITY AND PREDICTIVE VALUES
• For sensitivity and specificity, the reference variable (‘denominator) is the DISEASE
• For predictive value, the reference variable (‘denominator’) is the TEST
![Page 14: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/14.jpg)
METHOD 2: NATURAL FREQUENCIES TREE
Population
100
![Page 15: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/15.jpg)
IN EVERY 100 PEOPLE, 4 WILL HAVE THE DISEASE
Disease +
4
Disease -
96
Population
100
If these 100 people are representative of the population at risk, the assessed rate of those with the
disease (4%) represents the PREVALENCE of the disease – it can also be considered the PRE-TEST
PROBABILITY of having the disease
![Page 16: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/16.jpg)
OF THE 4 PEOPLE WITH THE DISEASE, THE TEST WILL DETECT 3
Disease +
4
Disease -
96
Test +
3
Test -
1
Population
100
In other words, the sensitivity is
75%
![Page 17: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/17.jpg)
AMONG THE 96 PEOPLE WITHOUT THE DISEASE, 7 WILL TEST POSITIVE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
In other words, the specificity is
93%
![Page 18: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/18.jpg)
POSITIVEPREDICTIVE
VALUE = 30%
AMONG THOSE WHO TEST POSITIVE, 3 IN 10 WILL ACTUALLY HAVE THE DISEASE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
This is also the POST-TEST PROB- ABILITY of having
the disease
![Page 19: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/19.jpg)
NEGATIVEPREDICTIVEVALUE = 99%
AMONG THOSE WHO TEST NEGATIVE, 89 OF 90 WILL NOT HAVE THE DISEASE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
![Page 20: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/20.jpg)
CONVERSELY, IF SOMEONE TESTS NEGATIVE, THE CHANCE OF HAVING THE DISEASE IS ONLY 1 IN 90
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
![Page 21: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/21.jpg)
PREDICTIVE VALUES AND CHANGING PREVALENCE
Disease +
4
Disease -
996
Population
1000
Prevalence reduced by an order of magnitude from 4%
to 0.4%
![Page 22: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/22.jpg)
PREDICTIVE VALUE AND CHANGING PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Sensitivity and Specificity
unchanged
![Page 23: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/23.jpg)
POSITIVEPREDICTIVEVALUE = 4%
POSITIVE PREDICTIVE VALUE AT LOW PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Previously, PPV was 30%
![Page 24: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/24.jpg)
NEGATIVEPREDICTIVEVALUE >99%
NEGATIVE PREDICTIVE VALUE AT LOW PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Previously, NPV was 99%
![Page 25: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/25.jpg)
PREDICTION OF LOW PREVALENCE EVENTS
• Even highly specific tests, when applied to low prevalence events, yield a high number of false positive results
• Because of this, under such circumstances, the Positive Predictive Value of a test is low
• However, this has much less influence on the Negative Predictive Value
![Page 26: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/26.jpg)
RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUE
0
0.2
0.4
0.6
0.8
1
0.05 0.2 0.4 0.6 0.8 0.95
Pre-test Probability (Prevalence)
Pre
dictive V
alu
e
PPVNPV
Based on a test with 90% sensitivity and 82% specificity
Difference between PPV and
NPV relatively small
Difference between PPV and
NPV relatively large
![Page 27: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/27.jpg)
RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUE
Based on a test with 75% sensitivity and 93% specificity
Prevalence
Pre
dic
tive V
alu
e
![Page 28: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/28.jpg)
PERFORMANCE OF A TEST WITH CHANGING PREVALENCE
A : Sensitivity = Specificity = 0.9LR+ = 9.0
B : Sensitivity = Specificity = 0.7LR+ = 3.0
C : Sensitivity = Specificity = 0.5LR+ = 1.0
PO
ST-T
ES
T P
RO
BA
BIL
ITY
![Page 29: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/29.jpg)
LIKELIHOOD
Disease +
4
Test +
3
Test -
1
Population
100
The likelihood that someone with the disease will have a positive test is ¾ or 75%This is the same as the sensitivity
![Page 30: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/30.jpg)
LIKELIHOOD II
Disease -
96
Test +
7
Test -
89
Population
100
The likelihood that someone without the disease will have a positive test is 7/96 or 7%This is the same as the (1-specificity)
![Page 31: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/31.jpg)
LIKELIHOOD RATIO
LIKELIHOOD OF POSITIVE TEST IN THE ABSENCE OF THE DISEASE
SENSITIVITY
1- SPECIFICITY= = 10.7
LIKELIHOOD OF POSITIVE TEST GIVEN THE DISEASE
=LIKELIHOOD RATIO
A Likelihood Ratio of 1.0 indicates an uninformative test (occurs when sensitivity and specificity are both
50%)
The higher the Likelihood Ratio, the better the test (other factors being equal)
0.75
0.07=
![Page 32: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/32.jpg)
METHOD 3: ‘TRADITIONAL’ 2x2 TABLES
![Page 33: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/33.jpg)
SENSITIVITY
SENSITIVITY
The proportion of people with the diagnosis (N=4) who are correctly identified (N=3)
Sensitivity = a/(a+c) = 3/4 = 75%
FALSE NEGATIVES
![Page 34: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/34.jpg)
SPECIFICITY
SPECIFICITY
The proportion of people without the diagnosis (N=96) who are correctly identified (N=89)
Specificity = d/(b+d) = 89/96 = 93%
FALSE POSITIVES
![Page 35: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/35.jpg)
PRE-TEST ODDS
In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)
![Page 36: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/36.jpg)
POST-TEST ODDS
In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS)
In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)
![Page 37: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/37.jpg)
POST-TEST ODDS
In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS)
In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)
In those who score negative on the test, the odds of having the disease are 1 to 89 or approximately 1%
![Page 38: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/38.jpg)
DIAGNOSTIC ODDS RATIO
The Diagnostic Odds Ratio is the ratio of odds of having the diagnosis given a positive test to those of having the diagnosis given a negative test
2.38011.0429.089
17
3DOR
Potentially useful as an overall summary measure, but only in conjunction with other measures (LR, sensitivity, specificity)
![Page 39: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/39.jpg)
BAYES THEOREM
POST-TEST ODDS =
LIKELIHOOD RATIO x PRE-TEST ODDS
![Page 40: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/40.jpg)
LIKELIHOOD RATIO AND PRE- AND POST-TEST PROBABILITIES
For a given test with a given likelihood ratio, the post-test probability will depend on the pre-test probability (that is, the prevalence of the condition in the sample being assessed)
![Page 41: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/41.jpg)
SENSITIVITY ANALYSIS OF A DIAGNOSTIC TEST
Value
95% CI
Pre-test probabilit
y35% 26% to 44%
![Page 42: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/42.jpg)
SENSITIVITY ANALYSIS OF A DIAGNOSTIC TEST
Applying the 95% confidence intervals above to the nomogram, the post-test probability is likely to lie in the range 55-85%
Value
95% CI
Pre-test probabilit
y35% 26% to 44%
Likelihood ratio
5.0 3.0 to 8.5
![Page 43: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/43.jpg)
RECEIVER OPERATING CHARACTERISTIC CURVE
Overall shape is predicted by the reciprocal relationship between sensitivity and specificity
The closer the curve gets to Sensitivity=1 and Specificity=1, the better the overall performance of the test
The diagonal line (representing Sensitivity=0.5 and Specificity=0.5) represents performance no better than chance
Hence the area under the curve gives a measure of the test’s performance
FALSE POSITIVE RATE (1-Specificity)
TR
UE P
OS
ITIV
E R
ATE
(Sen
sit
ivit
y)
![Page 44: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/44.jpg)
0
100
1-Specificity
Sensi
tivi
ty
AREA UNDER ROC CURVES
0
100
1-Specificity
Sensi
tivi
ty Sensitivity and specificity both 100% - TEST PERFECT
Sensitivity and specificity both 50% - TEST USELESS
AREA=1.0
AREA=0.5The area under a ROC curve will be between 0.5 and 1.0
![Page 45: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/45.jpg)
0
100
1-Specificity
Sensi
tivi
ty
AREA UNDER ROC CURVES
Area = 0.7 (between 0.5 and
1.0)
•Consider (hypothetically) two patients drawn randomly from the DISEASE+ and DISEASE- groups respectively
• If the test is used to guess which patient is from the DISEASE+ group, it will be right 70% of the time
![Page 46: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/46.jpg)
APPLYING A DIAGNOSTIC TEST IN DIFFERENT SETTINGS
• The Positive Predictive Value of a test will vary (according to the prevalence of the condition in the chosen setting)
• Sensitivity and Specificity are usually considered properties of the test rather than the setting, and are therefore usually considered to remain constant
• However, sensitivity and specificity are likely to be influenced by complexity of differential diagnoses and a multitude of other factors (cf spectrum bias)
![Page 47: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/47.jpg)
RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE
0102030405060708090
100
0 20 40 601-Specificity
Sensi
tivit
y
ACAT
MC
This study compared the performance of a dementia screening test in a community sample (ACAT) and a memory clinic sample (MC)
Flicker L, Loguidice D, Carlin JB, Ames D. The predictive value of dementia screening instruments in clinical populations. International Journal of Geriatric Psychiatry 1997 ; 12 : 203-209
![Page 48: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/48.jpg)
METHOD 4: A TEST WITH NORMALLY DISTRIBUTED VALUES
Negative Positive
Degree of ‘positivity’ on test
% o
f G
rou
p
DISEASED
NON-DESEASED
Test cut-off
Assessing the performance of the test assumes that these two
distributions remain constant. However, each
of them will vary (particularly through spectrum or selection
bias)
![Page 49: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/49.jpg)
CASESNON-CASES
PERFORMANCE OF A DIAGNOSTIC TEST
Negative Positive
Degree of ‘positivity’ on test
% o
f G
rou
p
DISEASED
NON-DESEASED
Test cut-offFALSE
NEGATIVESFALSE
POSITIVES
![Page 50: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/50.jpg)
MINIMISING FALSE NEGATIVES: A SENSITIVE TEST
Negative Positive
Degree of ‘positivity’ on test
% o
f G
rou
p
DISEASED
NON-DESEASED
Test cut-off
Cut-off shifted to minimise false negatives ie to optimise sensitivity
CONSEQUENCES:
- Specificity reduced
- A Negative result from a seNsitive test rules out the diagnosis - snNout
CASESNON-CASES
![Page 51: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/51.jpg)
MINIMISING FALSE POSITIVES: A SPECIFIC TEST
Negative Positive
Degree of ‘positivity’ on test
% o
f G
rou
p
DISEASED
NON-DESEASED
Test cut-off
Cut-off shifted to minimise false positives ie to optimise specificity
CONSEQUENCES:
- Sensitivity reduced
- A Positive result from a sPecific test rules in the diagnosis - spPin
![Page 52: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/52.jpg)
NON-CASESCASES
METHOD 5: USING SCALES WITH DIFFERENT CUT-OFFS
-20 -15 -10 -5 0 5 10 15 20 25
PATIENTS
>24
24
23
22
21
20
19
18
<18
TRUE
POSITIV
ES
TRUE
NEGATI
VES
FALS
E
POSITIV
ES
FALS
E
NEGATI
VES
B
DC
A
Sensitivity = A/A+C
Specificity = D/B+D
MM
SE S
core
Chosen cut-off
![Page 53: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/53.jpg)
NON-CASESCASES
INCREASING SENSITIVITY
-20 -15 -10 -5 0 5 10 15 20 25
PATIENTS
>24
24
23
22
21
20
19
18
<18
TRUEPOSITIVES
FALSENEGATIVES
B
DC
A
Sensitivity = A/A+C
Specificity = D/B+D
MM
SE S
core
In a seNsitive test, false Negatives are minimised
A negative result from a sensitive test rules out the diagnosis (snNnout)
![Page 54: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/54.jpg)
NON-CASESCASES
INCREASING SPECIFICITY
-20 -15 -10 -5 0 5 10 15 20 25
PATIENTS
>24
24
23
22
21
20
19
18
<18
TRUENEGATIVES
FALSEPOSITIVES
B
DC
A
Sensitivity = A/A+C
Specificity = D/B+D
MM
SE S
core
In a sPecific test, false Positives are minimised
A positive result from a specific test rules in the diagnosis (spPin)
![Page 55: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/55.jpg)
KEY REFERENCES
Sedlmeier P and Gigerenzer G. Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General. 130 (3):380-400, 2001.
Knotternus JA (ed). The Evidence Base of Clinical Diagnosis. London: BMJ Books, 2002.
Sackett DL, Haynes RB, Guyatt G, and Tugwell P. Clinical Epidemiology : A Basic Science for Clinical Medicine. Boston, Mass: Little, Brown & Co, 1991.
Loong TW. Understanding sensitivity and specificity with the right side of the brain. BMJ 2003: 327: 716-19.
![Page 56: TEACHING ABOUT DIAGNOSIS Tom Sensky Teaching EBM/EBMH St Hughs College Oxford September 2005.](https://reader030.fdocuments.net/reader030/viewer/2022020105/55164379550346a2698b46dc/html5/thumbnails/56.jpg)