Wanda Pletcher CJE Altoona Area Junior High School Altoona, Pennsylvania
Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.
-
Upload
taylor-lopez -
Category
Documents
-
view
224 -
download
1
Transcript of Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.
![Page 1: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/1.jpg)
Mark Pletcher6/9/2011
Prognostic and Genetic Tests
![Page 2: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/2.jpg)
An Example
“Mammaprint”
Gene expression profiling for Breast CA Grind up the tumor, extract RNA Incubate with a microarray of DNA
fragments to estimate expression for each gene
70 previously identified genes predict outcomes
![Page 3: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/3.jpg)
Van de Vijver et al. NEJM 2002;347(25):1999-2009
![Page 4: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/4.jpg)
An Example
“Mammaprint”
Pattern of expression correlates with disease-free and overall survival
![Page 5: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/5.jpg)
Van de Vijver et al. NEJM 2002;347(25):1999-2009
![Page 6: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/6.jpg)
An Example
“Mammaprint”
10-year probability of:
Survival Free of mets
“Good” pattern 95% 85%“Bad” pattern 55% 51%
Van de Vijver et al. NEJM 2002;347(25):1999-2009
![Page 7: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/7.jpg)
Outline
Prognostic vs. Diagnostic Tests Evaluating a Prognostic Test
Accuracy Utility
Genetic Tests (very briefly)
![Page 8: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/8.jpg)
Prognostic vs. Diagnostic Tests
How is a prognostic test different from a diagnostic test?
![Page 9: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/9.jpg)
Diagnostic Test Prognostic Test
Purpose
Chance Event Occurs to Patient
Study Design
Maximum Obtainable
AUROC
Identify Prevalent Disease
Predict Incident Disease/Outcome
Prior to Test After Test
Cross-Sectional Cohort
<1 (not clairvoyant)1 (gold standard)
Prognostic vs. Diagnostic Tests
![Page 10: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/10.jpg)
Prognostic vs. Diagnostic Tests
Classic prognosis:
Prediction of death after diagnosis of a disease
![Page 11: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/11.jpg)
Prognostic vs. Diagnostic Tests
Prognosis, broadly speaking:
Prediction of any future event Death or recurrence of cancer Stroke after presentation for TIA Peri-operative MI in surgical patients First MI in asymptomatic persons
![Page 12: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/12.jpg)
Prognostic vs. Diagnostic Tests
Prognosis vs. Diagnosis: A Spectrum
Grey areas Pre-clinical disease: Coronary calcium “Reversible” disease: Tiny lung CA Irreversible predisposition: Huntington’s
gene
![Page 13: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/13.jpg)
Prognostic vs. Diagnostic Tests
Prognostication ≠ Etiology
Risk factor Causes the disease Reducing it may prevent disease Confounding is crucial issue in observational studies
Risk marker (i.e., prognostic factor) Predicts the disease Need not be concerned about unmeasured
confounders Not all risk markers are risk factors…(e.g., CRP)
![Page 14: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/14.jpg)
Evaluating Prognostic Tests Test Performance
Association Discrimination Calibration Reclassification Pitfalls
Test Utility
![Page 15: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/15.jpg)
Evaluating Prognostic Tests
Association Is the marker associated with
development of the disease? Odds ratio, relative risk, hazard ratio “Independently associated” means after
adjustment for other known predictors
![Page 16: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/16.jpg)
Evaluating Prognostic Tests
HRadj = 4.6 P<.001
Van de Vijver et al. NEJM 2002;347(25):1999-2009
![Page 17: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/17.jpg)
Evaluating Prognostic Tests
Discrimination Ability to distinguish between people
with higher or lower risk of disease Metrics: just like diagnostic tests!?
Sensitivity/specificity ROC curves
![Page 18: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/18.jpg)
Evaluating Prognostic Tests
Mammaprint
Sensitivity = 28/30 = 93%Specificity = 41/83 = 49%
Mets <5yr No mets
![Page 19: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/19.jpg)
Coronary artery calcium Predictor of CHD events Adds discrimination AUROC .63.68
FRS = Framingham Risk ScoreCACS = Coronary Artery Calcium Score
Greenland et al. JAMA 2004;291(2):210-215
Evaluating Prognostic Tests
![Page 20: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/20.jpg)
Evaluating Prognostic Tests
Discrimination Results are specific to a particular
time point 5-year risk of metastases or death 90-day risk of stroke
![Page 21: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/21.jpg)
Evaluating Prognostic Tests
Discrimination
Different results at 5 years….
![Page 22: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/22.jpg)
Evaluating Prognostic Tests
Discrimination
…than at 10 years
![Page 23: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/23.jpg)
Evaluating Prognostic Tests
Discrimination Often 1 time point is most relevant or
easily communicated, but information is lost…
Can think of a “set” of discrimination statistics/ROC curves
Harell’s C-Statistic Integrated C-statistic for survival data Similar interpretation as AUROC
Harrell et al. Stat Med 1996;15(4):361-87.
![Page 24: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/24.jpg)
Evaluating Prognostic Tests
Calibration How close is predicted risk to actual
risk?
![Page 25: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/25.jpg)
Evaluating Prognostic Tests
Prognostic test results are often converted into absolute risk estimates Like post-test probabilities in
diagnosis Required for clinical interpretation Estimated directly in a longitudinal
study
![Page 26: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/26.jpg)
Evaluating Prognostic Tests
But absolute risk estimates can be “off” When derivation population different
than target population, etc
Framingham example
![Page 27: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/27.jpg)
D’Agostino et al. JAMA 2001;286(2):180-187
![Page 28: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/28.jpg)
Evaluating Prognostic Tests
Calibration is “orthogonal” to discrimination Awful discrimination but good
calibration Awful calibration but good
discrimination Miscalibration leads to worse
errors, but it’s easier to fix…
![Page 29: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/29.jpg)
Evaluating Prognostic Tests
Reclassification How often does the test lead to
reclassification across a treatment threshold?
i.e., how often might the test lead to a change in treatment?
CRP reclassification example
![Page 30: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/30.jpg)
Evaluating Prognostic Tests
Reclassification How often does the test lead to
reclassification across a treatment threshold?
Cook et al. Annals of Int Med 2006;145(1):21-29
![Page 31: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/31.jpg)
Evaluating Prognostic Tests
Reclassification metrics Net Reclassification Improvement
(NRI) Net % reclassified correctly
Depends on specified treatment thresholds/categories
![Page 32: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/32.jpg)
Evaluating Prognostic Tests
Pitfalls for prognostic test studies
Loss to follow-up and competing risks Especially problematic if loss is
“differential”
![Page 33: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/33.jpg)
Evaluating Prognostic Tests
Pitfalls for prognostic test studies
Bias if clinician knows the test result e.g. – persons with coronary calcium+
are: More likely to get revascularization More likely to get referred to ED if they have
chest pain
![Page 34: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/34.jpg)
Evaluating Prognostic Tests
Pitfalls for prognostic test studies
Overfitting Test will perform best in sample from
which it is derived More variables and “choices” more
danger of overfitting Gene expression arrays, proteomics
![Page 35: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/35.jpg)
Evaluating Prognostic Tests
Clinical Utility Does it improve health?
![Page 36: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/36.jpg)
Evaluating Prognostic Tests
Test Result
Better patient understanding of disease/risk
Healthier patient behaviors
Better clinical decisions
1
2
3
Better health
Pletcher et al. Circulation 2011;123;1116-1124
![Page 37: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/37.jpg)
Evaluating Prognostic Tests
Clinical Utility Cannot be estimated from test
performance metrics alone Need to understand downstream
consequences, including Benefits and harms of interventions based on
test result Harms from test itself Quality and length of life Costs
![Page 38: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/38.jpg)
Evaluating Prognostic Tests
Clinical Utility Can be estimated directly…
Randomized trial of test-and-treat strategy …or indirectly
Decision analysis/cost-effectiveness modeling
Same issues for diagnostic tests, and especially important when screening apparently healthy people…
Pletcher et al. Circulation 2011;123;1116-1124
![Page 39: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/39.jpg)
Genetic Tests
Potentially useful for mechanistic insight
Prognostic implications across individuals in a family
Otherwise, must meet same standards for prognostic utility as other tests Single gene studies often disappointing
![Page 40: Mark Pletcher 6/9/2011 Prognostic and Genetic Tests.](https://reader031.fdocuments.net/reader031/viewer/2022013100/55146ac4550346414e8b5d86/html5/thumbnails/40.jpg)
Key concepts For prognostic tests, an element of time
and chance remain (perfect test impossible)
Discrimination vs. Calibration Reclassification indices help us
understand how often a test might change management
Clinical utility depends on accounting for net benefits and harms (and costs)