1 Statistical Lesson on Screening Stephen E. Fienberg Statistics 36-149 Fall, 2001.
-
Upload
grant-chambers -
Category
Documents
-
view
219 -
download
0
description
Transcript of 1 Statistical Lesson on Screening Stephen E. Fienberg Statistics 36-149 Fall, 2001.
1
Statistical Lesson on Screening
Stephen E. FienbergStatistics 36-149
Fall, 2001
2
Screening Tests
What is Pr(HIV+)?
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
True HIV Status
3
Screening Tests
What is Pr(HIV+)? Pr(HIV+)= 260/100,000=0.00260.
True HIV Status
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
4
Error Rates for Tests
What is Pr(HIV+|Test Result +)?
True HIV StatusTest Results +ve -ve Totals
+ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
5
Error Rates for Tests
What is Pr(HIV+|Test Result +)?Pr(HIV+ | Test Result +) = 254/7,635 = 0.0333 ==>
False Positive Rate = Pr(HIV- | Test Result +) = 7,381/7,635 = 0.9667
True HIV Status
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
6
Error Rates for Tests
What is Pr(HIV+|Test Result -)?
True HIV Status
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
7
Error Rates for Tests
What is Pr(HIV+|Test Result -)?Pr(HIV+ | Test Result -) = 6/92,365 = 0.000065
False Negative Rate
True HIV Status
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
8
Validity-Reliability of Tests
• Reliability means repeatability across different times, places, subjects, and/or experimental conditions.– Test-retest reliability-- same subject and same measurement
procedure including examiner, equipoment etc., on two occasions.
– Inter-rater reliability--different examiners/judges.
• Validity means consistent association between test and truth in general population of interest.
9
How to Measure Validity?
• Problem:– False positive and false negative error rates depend
on the mix of true HIV+ and HIV- people in the population.
– In our example, they depend on the frequency of positive and negative test results and the base rate for HIV+!
– So if the base rate for HIV+ changes our assessment of validity does.
10
Example: Airport Security
• Screening of luggage using x-ray equipment in airport security.
• What do terms Reliability (test-retest, and inter-rater) and Validity mean?
11
Sensitivity and Specificity for Tests
• Sensitivity of test= Pr(Test Result +| HIV+) = 254/260 = 0.977
• Specificity of test= Pr(Test Result -| HIV-) = 92,359/99,740 = 0.926
True HIV Status
Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365
Totals 260 99,740 100,000
12
• Can plot rates on a graph:
Characteristics of Test
13
• To convey information on validity we use plot of “sensitivity” and “1 - specificity” for different versions of the test.
• Let’s see how to do this in an example.
Receiver Operating Characteristic Curve
14
15
16
17
18
19
20
21
22
23
ROC Analysis
• To measure the validity we connect the dots on ROC plot and compute the area under the curve:– perfect test has area 1.– “chance” test has area 0.5.
24
Back To Airport Security
• Does any of this have relevance to the issues of “Terrorist” screening and “Racial”screening for security purposes?
25
References
• Handout on Web from Rhiannon Weaver• Slides from Center for Biomedical
Informatics, UPMC:– http://www.cbmi.upmc.edu/courses/fall/intro/oct12