Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics...
-
Upload
gervase-griffith -
Category
Documents
-
view
219 -
download
0
Transcript of Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics...
Thoughts on Biomarker Discovery and Validation
Karla Ballman, Ph.D.
Division of Biostatistics
October 29, 2007
Outline
• General guidelines
• Objectives of screening studies
• Phase I: Pre-clinical exploratory studies
• Phase II: Clinical assay development for clinical disease
• Phase III: Retrospective longitudinal repository studies
• Phase IV: Prospective screening studies
• Phase V: Prevention/control studies
General Guidelines
• Biomarker / Marker• indicator of a particular disease state
of a patient• individual marker, panel of markers,
signature, etc.
• Clinical development processseries of well-defined steps from identification of a potentially useful biomarker through systematic evaluation of its clinical utility
General Guidelines (2)
• What is the intended final use of biomarker?• clinical versus population screen• stand-alone biomarker versus panel• asymptomatic versus symptomatic normals• diagnosis versus prognosis versus prediction
• Need discrete decision points: pursue or not
• Criteria• identify markers that have promise to be clinically
useful• assess the best methodology for clinical
evaluation of markers in question• confirm or validate that additional clinical utility is
gained by using marker compared to standard practice
Screening Study Objectives
• Non-invasive
• Inexpensive
• Secreted by disease tissue only
• Highly sensitive
• Highly specific
Most likely requires the use of multiple markers to obtain high sensitivity and specificity.
Pre-clinical Exploratory Studies
• Comparison of disease tissue versus non-disease tissue
• Identify unique disease characteristics that might lead to ideas for clinical assays
• immunohistochemistry• western blot• gene expression profiles• protein expression profiles• levels of circulating antibodies
Pre-clinical Exploratory Studies (2)
• Primary objectives• identify leads for potential useful
markers• prioritize identified leads
• Specimen selection• (case) disease tissue before
treatment• (control) non-disease tissue matched
to case samples
Pre-clinical Exploratory Studies (3)
• Primary outcome measure• biomarker value• assay reliability / reproducibility
• Analysis• binary
• TPR: true positive rate• FPR: false positive rate
• continuous• sensitivity (TPR)• specificity (1 – FPR)• ROC curve
0.0 0.2 0.4 0.6 0.8 1.0
False Positive Rate
0.0
0.2
0.4
0.6
0.8
1.0
Tru
e P
osi
tive R
ate
ROC curve for Marker
Pre-clinical Exploratory Studies (4)
• Analysis (2)• selection of candidate markers
• find all that are statistically significant
• rank based on summary statistic• confirmatory analysis
• training / test samples• cross-validation
Pre-clinical Exploratory Studies (5)
• Sample size considerations• number and relative prevalence of disease
subtypes• ability of markers to discriminate among
different disease subtypes• number of candidate markers under study• number of case / control samples• statistical methodology being used
Best to select sample sizes based on simulation studies.
Clinical Assay Development
• Develop (non-invasive) clinical assay
• Primary objectiveestimate the TPR and FPR (or ROC curve) of the clinical biomarker assay
• Other objectives• optimize assay performance• determine relationship between assay levels on
disease tissue and clinical specimen• assess patient/subject characteristics associated
with biomarker status (level) in control subjects• assess disease characteristics associated with
biomarker status (level) in case subjects
Clinical Assay Development (2)
• Specimen selection• case samples before treatment• control samples matched to case
samples
• Primary outcome measureresult of clinical marker assay
Clinical assay development (3)
• Analysis• estimate of TPR and FPR (or ROC
curve)• test of TPR is too low and/or FPR is
too high• select minimally acceptable FPR and
determine whether TPR is about the acceptable threshold
Clinical assay development (4)
• Sample Size• depends on the precision wanted for
TPR and FPR• choose size for adequate power to
determine whether TPR and FPR are acceptable
Retrospective (Longitudinal) Repository Studies
• Idea: compare the assay values of case samples collected before their diagnosis to control samples
• Primary objectives• evaluate, as a function of time before
clinical diagnosis, the capacity of the biomarker to detect preclinical disease
• define criteria for a positive screening test
Retrospective (Longitudinal) Repository Studies (2)
• Other objectives• explore the impact of covariates
(demographics, disease-related characteristics, etc.) on the discriminatory abilities of the biomarker before clinical diagnosis
• compare markers with a view to selecting those that are most promising
• to develop algorithms for screen positivity based on combinations of markers
• determine a screening interval if repeated screening is of interest
Retrospective (Longitudinal) Repository Studies (3)
• Specimen selection• should be protocol driven• cases/controls should be obtained
from target population• controls are those that develop
disease• should match on all variables,
including follow-up
Retrospective (Longitudinal) Repository Studies (4)
• Primary outcomeresult of clinical marker assay
• Analysis• comparison TPR and FDR (or ROC
curves)• consider restricting analysis to TPR at the
(maximally) acceptable FPR rate• ROC curves should be time-dependent (to
account for time from test to disease presentation)
Retrospective (Longitudinal) Repository Studies (5)
• Sample size• number of case subjects• number of control subjects• number of clinical specimens per subject
The sample sizes should ensure that, for each preclinical time lag of interest (e.g., 1 year, 2 years, 4 years), there are enough specimens from control subjects and from case subjects taken close to those intervals so that biomarker accuracy can be estimated with sufficient precision.
Prospective Screening Studies
• Idea: the screen is applied to individuals and definitive diagnostic procedures are applied at that time to those screening positive
• the number and nature of cases detected with the screening tool are determined
• as are the numbers of subjects falsely screening positive and referred for work-up
Prospective Screening Studies (2)
• Primary objectivedetermine the operating characteristics (TPR and FPR) of the biomarker based screening test in a relevant population
• Other objectives• describe the characteristics of disease detected
by screening test• assess practicability of applying the screening
program• make preliminary assessments of effects of
screening on costs/mortality/morbidity• monitor disease that occurs but is not detected by
the screening protocol
Prospective Screening Studies (3)
• Subject selection• target population• inclusion/exclusion criteria• also consider inclusion of unscreened
control arm
• Primary outcome measure• screening test positive and disease
confirmed• screening test positive and disease not
confirmed• screening test negative
Prospective Screening Studies (4)
• Analysis• estimate of detection rate: those screened
positive who are positive• estimate of false-referral rate: those
screened positive but do not have disease• multivariable analysis to adjust for
covariates• comparison of multiple screening tests
• Sample size• depends on desired precision, or • depends on relative performance if
comparing different screening assays
Disease Control Studies
• Idea: determine whether screening reduces the burden of disease on the population
• Primary objectiveestimate the reductions in disease mortality afforded by the screening test
Disease Control Studies (2)
• Other objectives• obtain information about the costs of
screening and treatment and the cost per life saved
• evaluate compliance with screening and work-up in a diverse range of settings
• compare different screening protocols and/or to compare different approaches to treating screen-detected subjects in regard to effects on mortality and costs
Disease Control Studies (3)
• Subject selection• randomly selected from populations
in which the screening program is likely to be implemented
• Ideal: standard parallel-arm randomized clinical trial, with one arm consisting of subjects undergoing the screening protocol and the other arm consisting of unscreened subjects
Disease Control Studies (4)
• Primary outcometime from entry into the study until death
• Analysis• survival analysis methods are used
to compare the study arms with regard to overall mortality
• methods for comparing costs and quality of life for randomized trials
Disease Control Studies (5)
• Sample sizeTo detect a 20% reduction in cause-specific mortality with 80% power at the .05 two-sided significance level, standard calculations indicate that 650 deaths would need to be observed
Discussion
• Not all studies need to undergo all the described phases
• Need for formal guidelines
• For Phase III studies (retrospective repository), need criteria to allocate scarce resources in sensible/fair fashion
• Choices of cases and controls in all phases is complex, requires thought
• Need new statistical methodology