Targeted Targeted (Enrichment) (Enrichment)

DesignDesign

Prospective Co-Prospective Co-Development of Drugs and Development of Drugs and

Companion DiagnosticsCompanion Diagnostics1.1. Develop a completely specified genomic Develop a completely specified genomic

classifier of the patients likely to benefit from a classifier of the patients likely to benefit from a new drugnew drug

• Pre-clinical, phase II data, archived specimens from Pre-clinical, phase II data, archived specimens from previous phase III studiesprevious phase III studies

2.2. Establish analytical validated test for the Establish analytical validated test for the classifierclassifier

3.3. Use the completely specified classifier to Use the completely specified classifier to design and analyze a new clinical trial to design and analyze a new clinical trial to evaluate effectiveness of the new treatment evaluate effectiveness of the new treatment with a pre-defined analysis plan that preserves with a pre-defined analysis plan that preserves the overall type-I error of the study.the overall type-I error of the study.

Guiding PrincipleGuiding Principle The data used to develop the classifier The data used to develop the classifier

should be distinct from the data used to should be distinct from the data used to test hypotheses about treatment effect test hypotheses about treatment effect in subsets determined by the classifierin subsets determined by the classifier Developmental studies can be exploratoryDevelopmental studies can be exploratory Studies on which treatment effectiveness Studies on which treatment effectiveness

claims are to be based should be definitive claims are to be based should be definitive studies that test a treatment hypothesis in studies that test a treatment hypothesis in a patient population completely pre-a patient population completely pre-specified by the classifierspecified by the classifier

Using phase II data, develop predictor of response to new drugDevelop Predictor of Response to New Drug

Patient Predicted Responsive

New Drug Control

Patient Predicted Non-Responsive

Off Study

Primarily for settings where the Primarily for settings where the classifier is based on a single gene classifier is based on a single gene whose protein product is the target whose protein product is the target of the drugof the drug eg Herceptin eg Herceptin

Evaluating the Efficiency of Evaluating the Efficiency of Strategy (I)Strategy (I)

Simon R and Maitnourim A. Evaluating the efficiency of Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004; Correction and Cancer Research 10:6759-63, 2004; Correction and supplement 12:3229, 2006supplement 12:3229, 2006

Maitnourim A and Simon R. On the efficiency of Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-targeted clinical trials. Statistics in Medicine 24:329-339, 2005339, 2005

Model for Two Treatments With Model for Two Treatments With Binary ResponseBinary Response

Molecularly targeted treatment TMolecularly targeted treatment TControl treatment CControl treatment C1-1- Proportion of test + patients Proportion of test + patientsppcc control response probability control response probabilityresponse probability for test + response probability for test + patients on T is (ppatients on T is (pcc + + 11))Response probability for test – Response probability for test – patients on T is (ppatients on T is (pcc + + 00) )

Untargeted TrialUntargeted Trial Compare outcome for treatment group T vs control Compare outcome for treatment group T vs control

group C without classifier datagroup C without classifier data Fisher-Exact test at two-sided level .05 comparing Fisher-Exact test at two-sided level .05 comparing

response proportion in control group to response response proportion in control group to response proportion in treatment groupproportion in treatment group

Number of responses in C group of n patients is Number of responses in C group of n patients is binomial B(n,pbinomial B(n,pcc))

Number of responses in T group is Number of responses in T group is B(n,(1-B(n,(1-)(p)(pcc++11)+ )+ (p(pcc++00))))

Determine n patients per treatment group for power Determine n patients per treatment group for power 1-1- Use Ury & Fleiss approximation Biom 36:347-51,1980.Use Ury & Fleiss approximation Biom 36:347-51,1980.

Targeted TrialTargeted Trial Compare outcome for treatment group T vs control Compare outcome for treatment group T vs control

group C for Assay positive patientsgroup C for Assay positive patients Fisher-Exact test at two-sided level .05 comparing Fisher-Exact test at two-sided level .05 comparing

response proportion in control group to response response proportion in control group to response proportion in treatment groupproportion in treatment group

Number of responses in C group of n patients is Number of responses in C group of n patients is binomial B(n,pbinomial B(n,pcc))

Number of responses in T group is Number of responses in T group is B(n,pB(n,pcc++11))

Determine nDetermine nTT patients per treatment group for patients per treatment group for power 1-power 1- Use Ury & Fleiss approximation Biom 36:347-51,1980.Use Ury & Fleiss approximation Biom 36:347-51,1980.

ApproximationsApproximations Observed response rate ~ N(p,p(1-Observed response rate ~ N(p,p(1-

p)/n)p)/n)

ppee(1-p(1-pee) ~ p) ~ pcc(1-p(1-pcc))

Number of Randomized Number of Randomized Patients RequiredPatients Required

Type I error Type I error Power 1-Power 1- for obtaining significance for obtaining significance

21 12( )c c e ee c

k kn p q p q

p p

Randomized RatioRandomized Ratio(normal approximation)(normal approximation)

RandRat = nRandRat = nuntargeteduntargeted/n/ntargetedtargeted

11= rx effect in test + patients= rx effect in test + patients 00= rx effect in test - patients= rx effect in test - patients =proportion of test - patients=proportion of test - patients If If 00=0, RandRat = 1/ (1-=0, RandRat = 1/ (1-) ) 22

If If 00= = 11/2, RandRat = 1/(1- /2, RandRat = 1/(1- /2)/2)22

2

1

1 0(1 )RandRat

Screened RatioScreened Ratio NNuntargeteduntargeted = n = nuntargeteduntargeted

NNtargeted targeted == nntargetedtargeted/(1-/(1-) )

ScreenRat = NScreenRat = Nuntargeteduntargeted/N/Ntargetedtargeted=(1- =(1- )RandRat)RandRat

No treatment Benefit for Test - No treatment Benefit for Test - PatientsPatients

nnstdstd / n / ntargetedtargeted

Proportion Test Proportion Test PositivePositive

RandomizedRandomized ScreenedScreened

0.750.75 1.781.78 1.331.33

0.50.5 44 22

0.250.25 1616 44

Treatment Benefit for Test – Treatment Benefit for Test – Pts Half that of Test + PtsPts Half that of Test + Pts

n nstdstd / n / ntargetedtargeted

Proportion Test Proportion Test PositivePositive

RandomizedRandomized ScreenedScreened

0.750.75 1.311.31 0.980.98

0.50.5 1.781.78 0.890.89

0.250.25 2.562.56 0.640.64

Relative efficiency of targeted design depends Relative efficiency of targeted design depends on on proportion of patients test positiveproportion of patients test positive effectiveness of new drug (compared to control) for effectiveness of new drug (compared to control) for

test negative patientstest negative patients When less than half of patients are test positive When less than half of patients are test positive

and the drug has little or no benefit for test and the drug has little or no benefit for test negative patients, the targeted design requires negative patients, the targeted design requires dramatically fewer randomized patientsdramatically fewer randomized patients

The targeted design may require fewer or more The targeted design may require fewer or more screened patients than the standard designscreened patients than the standard design

TrastuzumabTrastuzumabHerceptinHerceptin

Metastatic breast cancerMetastatic breast cancer 234 randomized patients per arm234 randomized patients per arm 90% power for 13.5% improvement in 1-90% power for 13.5% improvement in 1-

year survival over 67% baseline at 2-sided year survival over 67% baseline at 2-sided .05 level.05 level

If benefit were limited to the 25% test + If benefit were limited to the 25% test + patients, overall improvement in survival patients, overall improvement in survival would have been 3.375%would have been 3.375% 4025 patients/arm would have been required 4025 patients/arm would have been required

Treatment Hazard Treatment Hazard Ratio for Marker Ratio for Marker Positive PatientsPositive Patients

Number of Events for Number of Events for Targeted DesignTargeted Design

Number of Events for Traditional Number of Events for Traditional DesignDesign

Percent of Patients Marker Percent of Patients Marker PositivePositive

20%20% 33%33% 50%50%

0.50.5 7474 20402040 720720 316316

Comparison of Targeted to Untargeted Comparison of Targeted to Untargeted DesignDesign

Simon R,Simon R, Development and Validation of Biomarker Classifiers for Treatment Development and Validation of Biomarker Classifiers for Treatment Selection, JSPISelection, JSPI

Web Based Software for Web Based Software for Comparing Sample Size Comparing Sample Size

RequirementsRequirements

http://brb.nci.nih.gov http://brb.nci.nih.gov

““Stratification Stratification Design”Design”

Developmental Strategy (II)Developmental Strategy (II)

Develop Predictor of Response to New Rx

Predicted Non-responsive to New Rx

Predicted ResponsiveTo New Rx

ControlNew RX Control

New RX

Developmental Strategy (II)Developmental Strategy (II)

Do not use the test to restrict eligibility, but to Do not use the test to restrict eligibility, but to structure a prospective analysis planstructure a prospective analysis plan

Having a prospective analysis plan is essentialHaving a prospective analysis plan is essential “ “Stratifying” (balancing) the randomization is Stratifying” (balancing) the randomization is

useful to ensure that all randomized patients have useful to ensure that all randomized patients have tissue available but is not a substitute for a tissue available but is not a substitute for a prospective analysis planprospective analysis plan

The purpose of the study is to evaluate the new The purpose of the study is to evaluate the new treatment overall and for the pre-defined subsets; treatment overall and for the pre-defined subsets; not to modify or refine the classifier not to modify or refine the classifier

The purpose is not to demonstrate that repeating The purpose is not to demonstrate that repeating the classifier development process on independent the classifier development process on independent data results in the same classifierdata results in the same classifier

R Simon. Using genomics in clinical trial R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-design, Clinical Cancer Research 14:5984-93, 200893, 2008

R Simon. Designs and adaptive analysis R Simon. Designs and adaptive analysis plans for pivotal clinical trials of plans for pivotal clinical trials of therapeutics and companion diagnostics, therapeutics and companion diagnostics, Expert Opinion in Medical Diagnostics Expert Opinion in Medical Diagnostics 2:721-29, 20082:721-29, 2008

Validation of EGFR biomarkers for Validation of EGFR biomarkers for selection of EGFR-TK inhibitor therapy for selection of EGFR-TK inhibitor therapy for

previously treated NSCLC patientspreviously treated NSCLC patients

2nd line NSCLC

with specimen

FISHTesting

FISH +(~ 30%)

FISH −(~ 70%)

Erlotinib

Pemetrexed

Erlotinib

Pemetrexed

Outcome1° PFS2° OS, ORR

PFS endpointPFS endpoint 90% power to detect 50% PFS improvement in FISH+90% power to detect 50% PFS improvement in FISH+ 90% power to detect 30% PFS improvement in FISH90% power to detect 30% PFS improvement in FISH−−

Evaluate EGFR IHC and mutations as predictive markersEvaluate EGFR IHC and mutations as predictive markers Evaluate the role of RAS mutation as a negative Evaluate the role of RAS mutation as a negative

predictive markerpredictive marker

957 patients4 years accrual, 1196 patients

1-2 years minimum additional follow-up

Analysis Plan AAnalysis Plan A

Compare the new drug to the control Compare the new drug to the control for classifier positive patients for classifier positive patients If pIf p++>0.05 make no claim of effectiveness>0.05 make no claim of effectiveness If pIf p++ 0.05 claim effectiveness for the 0.05 claim effectiveness for the

classifier positive patients andclassifier positive patients and Compare new drug to control for classifier Compare new drug to control for classifier

negative patients using 0.05 threshold of negative patients using 0.05 threshold of significancesignificance

Sample size for Analysis Plan ASample size for Analysis Plan A

88 events in classifier + patients needed to 88 events in classifier + patients needed to detect 50% reduction in hazard at 5% two-detect 50% reduction in hazard at 5% two-sided significance level with 90% powersided significance level with 90% power

If 25% of patients are positive, then when If 25% of patients are positive, then when there are 88 events in positive patients there are 88 events in positive patients there will be about 264 events in negative there will be about 264 events in negative patientspatients 264 events provides 90% power for detecting 264 events provides 90% power for detecting

33% reduction in hazard at 5% two-sided 33% reduction in hazard at 5% two-sided significance level significance level

Analysis Plan BAnalysis Plan B

(Limited confidence in test)(Limited confidence in test)

Compare the new drug to the control overall Compare the new drug to the control overall for all patients ignoring the classifier.for all patients ignoring the classifier. If pIf poveralloverall 0.03 claim effectiveness for the eligible 0.03 claim effectiveness for the eligible

population as a wholepopulation as a whole Otherwise perform a single subset analysis Otherwise perform a single subset analysis

evaluating the new drug in the classifier + evaluating the new drug in the classifier + patientspatients If pIf psubsetsubset 0.02 claim effectiveness for the 0.02 claim effectiveness for the

classifier + patients.classifier + patients.

Sample size for Analysis Plan BSample size for Analysis Plan B

To have 90% power for detecting uniform To have 90% power for detecting uniform 33% reduction in overall hazard at 3% two-33% reduction in overall hazard at 3% two-sided level requires 297 events (instead of sided level requires 297 events (instead of 263 for similar power at 5% level)263 for similar power at 5% level)

If 25% of patients are positive, then when If 25% of patients are positive, then when there are 297 total events there will be there are 297 total events there will be approximately 75 events in positive patients approximately 75 events in positive patients 75 events provides 75% power for detecting 50% 75 events provides 75% power for detecting 50%

reduction in hazard at 2% two-sided significance reduction in hazard at 2% two-sided significance level level

By delaying evaluation in test positive patients, By delaying evaluation in test positive patients, 80% power is achieved with 84 events and 90% 80% power is achieved with 84 events and 90% power with 109 eventspower with 109 events

This analysis strategy is designed to This analysis strategy is designed to not penalize sponsors for having not penalize sponsors for having developed a classifier developed a classifier

It provides sponsors with an It provides sponsors with an incentive to develop genomic incentive to develop genomic classifiersclassifiers

Analysis Plan CAnalysis Plan C Test for difference (interaction) Test for difference (interaction)

between treatment effect in test between treatment effect in test positive patients and treatment effect positive patients and treatment effect in test negative patientsin test negative patients

If interaction is significant at level If interaction is significant at level intint then compare treatments separately then compare treatments separately for test positive patients and test for test positive patients and test negative patientsnegative patients

Otherwise, compare treatments overallOtherwise, compare treatments overall

Sample Size Planning for Sample Size Planning for Analysis Plan CAnalysis Plan C

88 events in test + patients needed to 88 events in test + patients needed to detect 50% reduction in hazard at 5% two-detect 50% reduction in hazard at 5% two-sided significance level with 90% powersided significance level with 90% power

If 25% of patients are positive, when there If 25% of patients are positive, when there are 88 events in positive patients there are 88 events in positive patients there will be about 264 events in negative will be about 264 events in negative patientspatients 264 events provides 90% power for detecting 264 events provides 90% power for detecting

33% reduction in hazard at 5% two-sided 33% reduction in hazard at 5% two-sided significance levelsignificance level

Simulation Results for Analysis Simulation Results for Analysis Plan CPlan C

Using Using intint=0.10, the interaction test has power =0.10, the interaction test has power 93.7% when there is a 50% reduction in 93.7% when there is a 50% reduction in hazard in test positive patients and no hazard in test positive patients and no treatment effect in test negative patientstreatment effect in test negative patients

A significant interaction and significant A significant interaction and significant treatment effect in test positive patients is treatment effect in test positive patients is obtained in 88% of cases under the above obtained in 88% of cases under the above conditionsconditions

If the treatment reduces hazard by 33% If the treatment reduces hazard by 33% uniformly, the interaction test is negative and uniformly, the interaction test is negative and the overall test is significant in 87% of casesthe overall test is significant in 87% of cases

Prospective-Retrospective Prospective-Retrospective StudyStudy

Prospective-Retrospective Prospective-Retrospective Evaluation of Prognostic or Evaluation of Prognostic or

Predictive ClassifierPredictive Classifier1.1. Analytically validate a single completely specified classifierAnalytically validate a single completely specified classifier2.2. Design a prospective clinical trial that definitvely addresses the Design a prospective clinical trial that definitvely addresses the

hypothesis of interest about the medical utility of the completely hypothesis of interest about the medical utility of the completely specified classifierspecified classifier1.1. Write a detailed protocol for the prospective study, including sample Write a detailed protocol for the prospective study, including sample

size justification and detailed statistical analysis plan addressing a size justification and detailed statistical analysis plan addressing a single hypothesis about the prognostic or predictive utility of a single single hypothesis about the prognostic or predictive utility of a single completely specified classifiercompletely specified classifier

3.3. Find a previously performed clinical trial that matches as closely Find a previously performed clinical trial that matches as closely as possible the prospective protocol developed aboveas possible the prospective protocol developed above1.1. Adequate designAdequate design2.2. Adequate sample size Adequate sample size 3.3. Adequate proportion of patients with archived tissue Adequate proportion of patients with archived tissue 4.4. Not used in any way in developing the classifier or analytically Not used in any way in developing the classifier or analytically

validating itvalidating it4.4. Perform the assay on the archived samples and then analyze the Perform the assay on the archived samples and then analyze the

data as defined in the prospective analysis plandata as defined in the prospective analysis plan

Use of Archived Specimens in Evaluation of Use of Archived Specimens in Evaluation of Prognostic and Predictive BiomarkersPrognostic and Predictive Biomarkers

Richard M. Simon, Soonmyung Paik and Daniel F. HayesRichard M. Simon, Soonmyung Paik and Daniel F. Hayes We propose modified guidelines for the conduct of reliable We propose modified guidelines for the conduct of reliable

analyses of prognostic and predictive biomarkers using analyses of prognostic and predictive biomarkers using archived specimens. These guidelines stipulate that: archived specimens. These guidelines stipulate that:

(i) archived tissue adequate for a successful assay must be (i) archived tissue adequate for a successful assay must be available on a sufficiently large number of patients from a available on a sufficiently large number of patients from a phase III trial that the appropriate analyses have adequate phase III trial that the appropriate analyses have adequate statistical power and that the patients included in the statistical power and that the patients included in the evaluation are clearly representative of the patients in the evaluation are clearly representative of the patients in the trial. trial.

(ii) The test should be analytically and pre-analytically (ii) The test should be analytically and pre-analytically validated for use with archived tissue.validated for use with archived tissue.

(iii) The analysis plan for the biomarker evaluation should be (iii) The analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier.evaluation of a single completely defined classifier.

iv) the results from archived specimens should be validated iv) the results from archived specimens should be validated using specimens from a similar, but separate, study. using specimens from a similar, but separate, study.

Use of Archived Specimens in Evaluation of Use of Archived Specimens in Evaluation of Prognostic and Predictive BiomarkersPrognostic and Predictive Biomarkers

Richard M. Simon, Soonmyung Paik and Daniel F. HayesRichard M. Simon, Soonmyung Paik and Daniel F. Hayes ConclusionsConclusions Claims of medical utility for prognostic and predictive Claims of medical utility for prognostic and predictive

biomarkers based on analysis of archived tissues can be biomarkers based on analysis of archived tissues can be considered to have either a high or low level of evidence considered to have either a high or low level of evidence depending on several key factors. depending on several key factors.

These factors include the analytical and pre-analytical These factors include the analytical and pre-analytical validation of the assay, the nature of the study from which the validation of the assay, the nature of the study from which the specimens were archived, the number and condition of the specimens were archived, the number and condition of the specimens, and the development prior to assaying tissue of a specimens, and the development prior to assaying tissue of a focused written plan for analysis of a completely specified focused written plan for analysis of a completely specified biomarker classifier. biomarker classifier.

Studies using archived tissues, when conducted under ideal Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the conditions and independently confirmed can provide the highest level of evidence. highest level of evidence.

Traditional analyses of prognostic or predictive factors, using Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a convenience sample of non analytically validated assays on a convenience sample of tissues and conducted in an exploratory and unfocused tissues and conducted in an exploratory and unfocused manner provide a very low level of evidence for clinical utility. manner provide a very low level of evidence for clinical utility.