Emerging Issues with Adaptation of Clinical Trial

Design in Drug Development*

H.M. James HungDivision of Biometrics I, Office of

Biostatistics, OPaSS, CDER, FDA

Presented in FDA/Industry Workshop, Bethesda, Maryland, September 16, 2005

*The view expressed here is not necessarily of the U.S. Food and Drug Administration.

J.Hung, 2005 FDA/Industry Workshop2

Contributors

Sue-Jane Wang (OB/OPaSS/CDER/FDA)Robert O’Neill (OB/OPaSS/CDER/FDA)John Lawrence (DB1/OB/OPaSS/CDER/FDA)

AcknowledgmentThanks are due to Charles Anello for valuable comments. This presentation is partly based on the research, supported by FDA/CDER RSR funds, #05-02, #05-14, 04-06, 02-06, 99/00-008, 96-010A.

J.Hung, 2005 FDA/Industry Workshop3

Outline

• Design consideration

• Logistics issues

• Statistical design/analysis issues

• Regulatory experiences

• Utility and pitfalls of design adaptation

• Summary

J.Hung, 2005 FDA/Industry Workshop4

Design Consideration

As a guiding principle, for Phase III trial, fixed information design without any possibility of early stopping is preferred.• minimize operational bias• valid statistical test available• unbiased estimator of treatment effect available• valid confidence interval available

These are the keys to the quality of statistical inference vital to ability of drawing conclusion from trial data.

J.Hung, 2005 FDA/Industry Workshop5

Design Consideration

Flexible designs Group-sequential design adaptive design Phase 2/3 combination design …………..

Utility of flexible designs in Phase II trials is probably obvious, at least for exploratory purpose, without subjecting to too many false positives.

J.Hung, 2005 FDA/Industry Workshop6

Design Consideration

Example: Surrogate markers are used in 1st phase of a clinical trial for predicting clinical effect or changing design in 2nd phase, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient populationClinical benefits are assessed at the end (second phase) of the trial where design specifications may have been modified.

J.Hung, 2005 FDA/Industry Workshop7

Design Consideration

When flexible design is considered, we need to ask:• How to proceed logistically? - standard operation procedure (SOP) - ability to comply with SOP - responsibility of every participant• Are trial conducts reviewable by regulatory agency?

Goal: minimize operational bias

J.Hung, 2005 FDA/Industry Workshop8

Design Consideration

When flexible design is considered, we need to ask:• Is valid statistical test available?• Is unbiased estimator of treatment effect available?• Is valid confidence interval available?

J.Hung, 2005 FDA/Industry Workshop9

Logistics Issues

Standard Operation Procedure• charters of all parties including independent thirty party (e.g., DSMB, CRO)• who see what?• how to ensure compliance with SOP?• how to minimize possible influence of adaptation on investigator/patient behavior? • will knowledge of external trials have adverse influence on the current trial?

J.Hung, 2005 FDA/Industry Workshop10

Statistical Design/Analysis Issues

• Much research has provided valid statistical methods for analysis of primary endpoint in flexible design trials - what point estimate and interval estimate to report in drug label needs more attention (have concern about using these estimates for future superiority/NI trials)

J.Hung, 2005 FDA/Industry Workshop11

Statistical Design/Analysis Issues

• Unclear how to best analyze important secondary endpoints after primary endpoint is positive• Unclear whether validity of multiplicity adjustment still holds• Unclear how to incorporate design efficiency consideration into entire drug development program - involve prediction of benefit/risk ratio - per-trial statistical efficiency is insufficient

J.Hung, 2005 FDA/Industry Workshop12

Regulatory Experience #1

In an oncology trial, time to progression (TTP) was a primary surrogate endpoint and overall survival (OS) was the ultimate primary endpoint. Group-sequential design was employed with an alpha-spending function for TTP.

After TTP was highly significant at an interim time and the trial was stopped, how to test OS given that only 20% of the planned total deaths is available?- May use the spending function of TTP for OS

J.Hung, 2005 FDA/Industry Workshop13

Regulatory Experience #2

In a heart failure trial, the primary endpoint is a composite endpoint of clinical outcomes (death, MI, stroke, etc.). Sample size and total # of events are increased, based on a new projected effect size from the data of an interim analysis.

How to test the component endpoints and other secondary endpoints?- relationship in location parameters of and correlation between the endpoints need to be considered

J.Hung, 2005 FDA/Industry Workshop14

Regulatory Experience #3

A trial is designed to study mortality endpoint.It is then divided into two subtrials to study a non-mortal endpoint (e.g., quality of life). - alpha allocation - replication concept - complication with interim analysis - p-value adjustment for the non-mortal endpoint - how to handle competing-risk problem

J.Hung, 2005 FDA/Industry Workshop15

Regulatory Experience #4

Surrogate markers or biomarkers are used in early stages for predicting drug effect or changing design elements in later stages, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient populationClinical benefits are assessed at the end of the trial whose design specifications may havebeen modified.

J.Hung, 2005 FDA/Industry Workshop16

Regulatory Experience #4 <cont’d>

Q: Can clinical endpoint data from exploratory stages be combined with Phase III data in analysis for confirmatory purpose?If yes, how to proceed in analysis?- Weighted combination Z statistic and the corresponding point estimate and CI can be useful tools. Need to prespecify the weight !- Concept of overall type I error can be unclear- Combining two stages at best results in one study

J.Hung, 2005 FDA/Industry Workshop17

Regulatory Experience #4 <cont’d>

- Issue with # of hypotheses to explore in Phase II (pertaining to statistical control of total type I errors)? No (or ignored) if Phases II and III are not combined Yes (needing attention) in Phases II and III are combined This makes little sense. [Hung, Wang, O’Neill (2004)]

How to proceed logistically?

J.Hung, 2005 FDA/Industry Workshop18

Utility and Pitfalls of Design Adaptation

Major pitfalls concern compromise of integrity of trial conduct.

Major utility is potential saving of a trial if the planning assumptions fall short. Flexibility can enhance the probability of trial success with integrity protected if properly planned and complied.

J.Hung, 2005 FDA/Industry Workshop19

Utility and Pitfalls of Design Adaptation

For statistical analysis, with proper determination of weighting, a weighted combination of Z statistics can be attractive and powerful.

III ZwZwZ 1

For any fixed weight, Z is valid under the nullhypothesis. A consistent estimator and valid confidence interval for the parameter Z tests are available to draw conclusion consistent with that from Z. How to find a good w?

J.Hung, 2005 FDA/Industry Workshop20

Utility and Pitfalls of Design Adaptation

If the weight w is random, then with properly adjusted critical value, Z can be valid. Q: Are point estimator and CI available to enable us to draw conclusion consistent with that from Z?

J.Hung, 2005 FDA/Industry Workshop21

Utility and Pitfalls of Design Adaptation

Don’t rely too much on the conditional power evaluated at the interim observed estimate or any data observed at too early interim looks- planning a trial conservatively is still needed.

J.Hung, 2005 FDA/Industry Workshop22

Summary

• More careful planning is necessary for flexible design (group-sequential or adaptive) in regulatory applications

- operational bias is a critical issue

• Conventional statistical methods need tune up in use of flexible designs

- adjust critical value of statistical test

- find valid point estimate and CI

J.Hung, 2005 FDA/Industry Workshop23

Summary

• Analysis methods suiting flexible designs may be preferable to properly adjusted conventional methods

- need to explain the differences to our

clients (can be very difficult)

- still need valid point estimate and CI

J.Hung, 2005 FDA/Industry Workshop24

Summary

• Too much adaptation should be discouraged

- what hypothesis to test becomes unclear - protecting integrity of trial is difficult - what infrastructures are needed for logistics is still quite unknown - reviewability by regulatory agency is unknown and very difficult

J.Hung, 2005 FDA/Industry Workshop25

Selected References

Bauer, Köhne (1994, Biometrics)Bauer, Röhmel (1995, Stat. In Med.)Lan, Trost (1997, ASA Proceedings)Proschan, Hunsberger (1995, Biometrics)Fisher (1998, Stat. In Med.)Shen, Fisher (1999, Biometrics) Cui, Hung, Wang (1999, Biometrics)Bauer, Kieser (1999, Statistics in Medicine)Posch, Bauer (1999, Biometrical J.)Kieser, Bauer, Lehmacher (1999, Biometrical J.)Lehmacher, Wassmer (1999, Biometrics)Müller & Schäfer (2001, Biometrics)Hellmich (2001, Biometrics)Liu & Chi (2001, Biometrics)

J.Hung, 2005 FDA/Industry Workshop26

Bauer, Brannath, Posch (2002, Method Inform Med)Brannath, Posch, Bauer (2002, JASA) Cui (2002, Encyclopedia of Biopharmaceutical Statistics)Li, Shih, Xie, Lu (2002, Biostatistics)Lawrence (2002, J. of Biopharmaceutical Statistics)Lawrence (2002, J. of Pharmaceutical Statistics)Lawrence & Hung (2002, Biometrical J.)Jennison & Turnbull (2003, Statistics in Medicine)Tsiatis & Mehta (2003, Biometrika)Liu, Proschan, Pledger (2003, JASA)Proschan, Liu, Hunsberger (2003, Statistics in Medicine)Wang, Hung, O’Neill (2004, ASA Proceedings)Hung, Wang, O’Neill (2004, ASA talk)Liu (2002, 2003, 2004, ASA talks, other talks)Lan (2000-2005 ASA, ICSA, FDA talks)Mehta (2004, talk on FDA-MIT workshop)

J.Hung, 2005 FDA/Industry Workshop27

Hung, Wang, O’Neill (2004, BASS talk)Lan, Soo, Siu, Wang (2005, J. of Biopharmaceutical Statistics)Hung, Cui, Wang, Lawrence (2005, J. of Biopharmaceutical Statistics)Hung (2005, WNAR talk)Wang, Hung (2005, ASA talk)Liu (2005, WNAR talk)