Slide 1

1 Bayesian Clinical Trials Scott M. Berry scott@berryconsultants.com Scott M. Berry scott@berryconsultants.com Slide 2 2 Bayesian Statistics Reverend Thomas Bayes (1702-1761) Essay towards solving a problem in the doctrine of chances (1764) This paper, on inverse probability, led to Bayes theorem, which led to Bayesian Statistics Slide 3 3 Bayes Theorem Bayesian inferences follow from Bayes theorem: '( | X) ( )*f (X | ) Assess prior ; subjective, include available evidence Construct model f for data Find posterior ' Bayesian inferences follow from Bayes theorem: '( | X) ( )*f (X | ) Assess prior ; subjective, include available evidence Construct model f for data Find posterior ' Slide 4 4 Simple Example Coin, P(HEADS) = = 0.25 or =0.75, equally likely. DATA: Flip coin twice, both heads. ??? Slide 5 5 Bayes Theorem Pr[ = 0.75 | DATA] = Pr[DATA | p=0.75] Pr[ =0.75] Pr[DATA | =0.75] Pr[ =0.75]+ Pr[DATA | =0.25] Pr[ =0.25] ----------------------------------------------------------------------------------- (0.75) 2 (0.5) (0.75) 2 (0.5) + (0.25) 2 (0.5) ----------------------------------- = 0.90 Likelihood Prior Probabilities Posterior Probabilities Slide 6 6 Rare Disease Example Suppose 1 in 1000 people have a rare disease, X, for which there is a diagnostic test which is 99% effective. A random subject takes the test, which says POSITIVE. What is the probability they have X? (0.99) (0.001) (0.99) (0.001) + (0.01) (0.999) --------------------------------------- = 0.0902 !!! Slide 7 7 Bayesian Statistics A subjective probability axiomatic approach was developed with Bayes theorem as the mathematical crank-- Savage, Lindley (1950s) Very different than classical statistics: a collection of tools Before 1980-1990?: A philosophical niche, calculation very hard. Early 1990s: Computers and methods made calculation possibleand more! Slide 8 8 Bayesian Approach Probabilities of unknowns: hypotheses, parameters, future data Hypothesis test: Probability of no treatment effect given data Interval estimation: Probability that parameter is in the interval Synthesis of evidence Tailored to decision making: Evaluate decisions (or designs), weigh outcomes by predictive probabilities Probabilities of unknowns: hypotheses, parameters, future data Hypothesis test: Probability of no treatment effect given data Interval estimation: Probability that parameter is in the interval Synthesis of evidence Tailored to decision making: Evaluate decisions (or designs), weigh outcomes by predictive probabilities Slide 9 9 Frequentist vs. Bayesian Seven comparisons 1. Evidence used? 2. Probability, of what? 3. Condition on results? 4. Dependence on design? 5. Flexibility? 6. Predictive probability? 7. Decision making? 1. Evidence used? 2. Probability, of what? 3. Condition on results? 4. Dependence on design? 5. Flexibility? 6. Predictive probability? 7. Decision making? Slide 10 10 Consequence of Bayes rule: The Likelihood Principle The likelihood function L X ( ) = f( X | ) contains all the information in an experiment relevant for inferences about The likelihood function L X ( ) = f( X | ) contains all the information in an experiment relevant for inferences about Slide 11 11 Short version of LP: Take data at face value But data can be deceptive Caveats... How identified? Why are they showing me this? Short version of LP: Take data at face value But data can be deceptive Caveats... How identified? Why are they showing me this? Slide 12 12 Example Data: 13 A's and 4 B's Parameter = = P(A wins) Likelihood 13 (1 ) 4 Frequentist conclusion? Depends on design Data: 13 A's and 4 B's Parameter = = P(A wins) Likelihood 13 (1 ) 4 Frequentist conclusion? Depends on design Slide 13 13 Frequentist hypothesis testing P-value = Probability of observing data as or more extreme than results, assuming H 0. P-V = P(tail of dist. | H 0 ) Four designs: (1) Observe 17 results (2) Stop trial once both 4 A's and 4 B's (3) Interim analysis at 17, stop if 0 - 4 or 13 - 17 A's, else continue to n = 44 (4) Stop when "enough information" P-value = Probability of observing data as or more extreme than results, assuming H 0. P-V = P(tail of dist. | H 0 ) Four designs: (1) Observe 17 results (2) Stop trial once both 4 A's and 4 B's (3) Interim analysis at 17, stop if 0 - 4 or 13 - 17 A's, else continue to n = 44 (4) Stop when "enough information" Slide 14 14 Design (1): 17 results Binomial distribution with n = 17, = 0.5; P-value = 0.049 Binomial distribution with n = 17, = 0.5; P-value = 0.049 Slide 15 15 Design (2): Stop when both 4 As and 4 Bs Two-sided negative binomial with r = 4, = 0.5; P-value = 0.021 Two-sided negative binomial with r = 4, = 0.5; P-value = 0.021 Slide 16 16 Design (3): Interim analysis at n=17, possible total is 44 Analyses at n = 17 & 44; stop @ 17 if 0-4 or 13-17; P = 0.085 Analyses at n = 17 & 44; stop @ 17 if 0-4 or 13-17; P = 0.085 Both shaded regions = 0.049 P(both) = 0.013; net = 2(0.049) 0.013 = 0.085 P(both) = 0.013; net = 2(0.049) 0.013 = 0.085 Slide 17 17 Design (4): Scientists stopping rule: Stop when you know the answer Cannot calculate P-value Strictly speaking, frequentist inferences are impossible Cannot calculate P-value Strictly speaking, frequentist inferences are impossible Slide 18 18 Bayesian Calculations Data: 13 A's and 4 B's Parameter = = P(A wins) For ANY design with these results, the likelihood function is P(data | p) 13 (1 ) 4 Posterior probabilities & Bayesian conclusion same for any design Data: 13 A's and 4 B's Parameter = = P(A wins) For ANY design with these results, the likelihood function is P(data | p) 13 (1 ) 4 Posterior probabilities & Bayesian conclusion same for any design Slide 19 19 Likelihood function of Slide 20 20 Posterior Distribution Prior: 1 0 < < 1 Posterior 1 * 13 (1 ) 4 = 1 * 13 (1 ) 4 / 1 * 13 (1 ) 4 d = {13!4!/18!} 13 (1 ) 4 Prior: 1 0 < < 1 Posterior 1 * 13 (1 ) 4 = 1 * 13 (1 ) 4 / 1 * 13 (1 ) 4 d = {13!4!/18!} 13 (1 ) 4 Slide 21 21 Posterior density of for uniform prior: Beta(14,5) Slide 22 22 Pr[ > 0.5 ] Slide 23 23 PREDICTIVE PROBABILITIES Distribution of future data? P(next is an A) = ? Critical component of experimental design In monitoring trials Slide 24 24 Laplaces rule of succession P(A wins next pair | data) = EP(A wins next pair | data, ) = E( | data) = mean of Beta(14, 5) = 14/19 Laplace uses Beta(1,1) prior Slide 25 25 Updating w/next observation Slide 26 26 Suppose 17 more observations P(A wins x of 17 | data) = EP(A wins x | data, ) = Beta-Binomial Distribution Slide 27 27 Predictive distribution Predictive distribution of # of successes in next 17 tries: Has more variability than any binomial Has more variability than any binomial 88% probability of statistical significance Slide 28 28 Best fitting binomial vs. predictive probabilities Binomial, p=14/19 Predictive, p ~ beta(14,5) 88% probability of statistical significance 96% probability of statistical significance Slide 29 29 Possible Calculation Simulate a from the beta(14,5) Simulate an x from binomial(17, ) Distribution of xs is beta-binomial--the predictive distribution Slide 30 30 Posterior and Predictivesame? Clinical Trial, 100 subjects. H A : > 0.25? FDA will approve if # success 33 [post > 0.95, beta(1,1)] See 99 subjects, 32 successes Pr[ > 0.25 | data ] = 0.955 Predictive prob trial success = 0.327 Slide 31 31 Predictive Probabilities for Medical Device Bayesian calculations FDA: Some patients have reached 2 years Some patients have only 1-yr follow- up Slide 32 32 Continuous data; Patients w/both 12 and 24 months Slide 33 33 Some patients with only 12-month data Slide 34 34 Kernel density estimates Slide 35 35 Small bandwidth (0.2 ) Slide 36 36 Larger bandwidth (0.3 ) Slide 37 37 Still larger bandwidth (0.4 ) Slide 38 38 Very large bandwidth (0.5 ) (nearly bivariate normal) Slide 39 39 Condition on 12-month value Slide 40 40 Conditional distribution of 24-month value (0.2 ) Slide 41 41 For largest bandwidth (0.5 ) Slide 42 42 Multiple imputation: simulate full set of 24-month data Slide 43 43 Simulate experimental patients and controls in this way multiple imputation Make inferences with full data (for example, equivalent improvement) Repeat simulations (10,000 times) Gives probability of future results for example, of equivalence Simulate experimental patients and controls in this way multiple imputation Make inferences with full data (for example, equivalent improvement) Repeat simulations (10,000 times) Gives probability of future results for example, of equivalence Slide 44 44 Monitoring example: Baxters DCLHb Diaspirin Cross-Linked Hemoglobin Blood substitute; emergency trauma Randomized controlled trial (1996+) Treatment: DCLHb Control: saline N = 850 (= 425x2) Endpoint: death Slide 45 45 Waiver of informed consent Data Monitoring Committee First DMC meeting: DCLHbSaline Dead 21 (43%) 8 (20%) Alive2833 Total 49 41 No formal interim analysis Slide 46 46 Bayesian predictive probability of future results (no stopping) Probability of significant survival benefit for DCLHb after 850 patients: 0.00045 (PP=0.0097) DMC paused trial: Covariates? DMC stopped the trial Slide 47 47 Herceptin in Neoadjuvant BC Endpoint: tumor response Balanced randomized, A & B Sample size planned: 164 Interim results after n = 34: Control: 4/16 = 25% (pCR) Herceptin: 12/18 = 67% (pCR) Not unexpected (prior?) Predictive prob of stat sig: 95% DMC stopped the trial ASCO and JCOreactions Slide 48 48 Mixtures: Data: 13 A's and 4 B's Likelihood p 13 (1p) 4 Slide 49 49 Mixture Prior ~ 0 I[p=p 0 ] + (1 0 ) Beta(, ) 0 I 0 p 0 13 (1 p 0 ) 4 + (1 0 ) Kp +13 1 (1 p) +4 1 ~ 0 I 0 + (1 0 ) Beta(, ) 0 p 13 (1 p) 4 0 = --------------------------------------------------- 0 p 13 (1 p) 4 + (1 0 ) ( ) ( ) ( + +17) -------------------------- ( ) ( ) ( + ) Slide 50 50 Pr(p=0.5) = 0.246 P(p > 0.5) = 0.742 Mixture Posterior 0 =.5 Slide 51 51 Crooked-Penny Example Flip the coin 20 times. What is for your coin? Everyone reports p for their coin. ^ A new estimate for ? Are others relevant for you? A new estimate for ? Are others relevant for you? Slide 52 52 Numbers of heads This is you Slide 53 53 One-Sample Problem ~ Beta( [X] ~ Binomial(n, ) [ X]~Beta( +X, +n-X) Mean = ( + X n) Slide 54 54 Prior: ~ Beta(1, 1 Posterior: ~ Beta(17, 5 0.77 For uniform prior ( = = 1) Slide 55 55 Prior: ~ Beta(10, 10 Posterior: ~ Beta(26, 14 0.65 For = = 10 Prior: ~ Beta(10, 10 Slide 56 56 Remember the other coins... This is you Slide 57 57 Learning about the prior In your setting the other coins give you information about the priorwhich helps!!!! The coins do not have to be the same or close, you learn the appropriate amount of borrowing. Slide 58 58 HIERARCHICAL MODELING Population: Sample: Sample from sample: Inferential problems problemsInferential Slide 59 59 Selecting coins Population of coinspopulation of s: Select two coins and toss each coin 10 times: one 9 heads, other 4 heads. Estimate 1, 2. Estimate 1, 2. Estimate distribution of s in population. Estimate distribution of s in population. Slide 60 60 Generic example: Unit is lab or drug variation or lot or study Unit s n s/n 1 20 20 1.00 2 4 10 0.40 3 11 16 0.69 4 10 19 0.53 5 5 14 0.36 6 36 46 0.78 7 9 10 0.90 8 7 9 0.78 9 4 6 0.67 Total 106 150 0.71 n = #observations s = #successes s/n = success proportion proportion Slide 61 61 If 1 = 2 =... = 9 = (all 150 units exchangeable) Slide 62 62 Assuming equal s, 95% CI for : (0.63, 0.77) But 7 of 9 estimates lie outside this interval. Combined analysis unsatisfactory. Nine different analyses even worse: nine individual CIs? Slide 63 63 Suppose n i independent observations on unit i Suppose each unit has its own, with 1,..., 9 having distribution G. Observe x's, not 's. X i ~ binomial(n i, i ). Likelihood is product of likelihoods of i Slide 64 64 Bayesian view: G unknown = G has probability distribution Prior distribution reflects heterogeneity vs homogeneity. Assume G is Beta(a,b), a > 0, b > 0 with a and b unknown. Study heterogeneity: little if a+b is large lots if a+b is small Slide 65 65 Beta(a,b) for a, b = 1, 2, 3, 4: Slide 66 66 Suppose uniform prior for a & b on integers 1,..., 10 Slide 67 67 Posterior probabilities for a & b Slide 68 68 Calculating posterior distribution of G Direct in this example Can be more complicated, and require: Gibbs sampling (BUGS) Other Markov chain Monte Carlo Slide 69 69 Posterior mean of G (also predictive density for ) Slide 70 70 Contrast with likelihood assuming all ps equal Slide 71 71 Bayesian questions: P( > 1/2) = ???? P(next unit in study i is success) = ? How to weigh results in unit i? How to weigh results in unit j? P(unit in 10th study is success) = ? How to weigh results in study i? Slide 72 72 Bayes estimates Unit x n x/n Bayes 1 20 20 1.00 0.90 2 4 10 0.40 0.53 3 11 16 0.69 0.69 4 10 19 0.53 0.57 5 5 14 0.36 0.48 6 36 46 0.78 0.77 7 9 10 0.90 0.80 8 7 9 0.78 0.73 9 4 6 0.67 0.68 Total 106 150 0.68 0.68 (0.71) Slide 73 73 Bayes estimates are regressed or shrunk toward overall mean Bayes estimates Unadjusted estimates Slide 74 74 Baseball Example 446 players in 2000 with > 100 at bats Jose Vidro Slide 75 75 How good was Jose Vidro? (200 hits in 606 at bats, 0.330) X ~ Binomial(606, JV X ~ Binomial(606, JV (hits) JV Beta( JV Beta( Slide 76 76 Empirical Bayes: EB EB mean = 0.269; var = 0.036 2 -.027 2 ) |X] ~ Beta(200+95.5, 406+258.9) (approx) Posterior mean = 0.308 Posterior st. dev. = 0.015 Slide 77 77 Science, Feb 6, 2004, pp 784-6 Slide 78 78 Efficacy of Pravastatin + Aspirin: Meta-Analyses [For statistical analysis, S.M. Berry et al., Journal of the American Statistical Association, 2004] www.fda.gov/ ohrms/dockets/ac/02/slides/ 3829s2_03_Bristol-Meyers-meta-analysis.ppt www.fda.gov/ ohrms/dockets/ac/02/slides/ 3829s2_03_Bristol-Meyers-meta-analysis.ppt Slide 79 79 Trial LIPID CARE REGRESS PLAC I PLAC II Totals Number of Subjects*% on Aspirin 82.7 83.7 54.4 67.5 42.7 80.4 Primary Endpoint CHD mortality CHD death & non-fatal MI Atherosclerotic progression (& events) 9014 4159 885 408 151 14,617 Atherosclerotic progression (& events) *99.7% of pravastatin-treated subjects received 40mg dose Meta-Analysis of these Pravastatin Secondary Prevention Trials Slide 80 80 Trial Commonalities Similar entry criteria Patient populations with clinically evident CHD Same dose of pravastatin (40mg) Randomized comparison against placebo All trials with durations of 2 years Pre-specified endpoints Covariates recorded Common meta-analysis data management Slide 81 81 Patient Group Comparisons PlaceboPravastatin Aspirin Users Aspirin Non-Users Prava+ASA Prava alone Placebo+ASA Placebo alone Randomized Groups Randomized Comparison Observational Comparison Slide 82 82 Is Pravastatin+Aspirin More Effective than Pravastatin Alone? Aspirin studies were conducted before statins were widely used Placebo-controlled trial with aspirin is not feasible Investigation of pravastatin database to explore this question Slide 83 83 Is the Combination More Effective than Pravastatin Alone? Unadjusted event rates in LIPID and CARE suggest pravastatin + aspirin is more effective than pravastatin alone Slide 84 84 Event Rates for Primary Endpoints in LIPID and CARE Aspirin Users Aspirin Non-Users 5.8% 8.8%14.8% 9.3% LIPID CHD Death CARE CHD Death or Non-fatal MI Pravastatin-treated Subjects Only Trial: Primary Endpoint: Observational Comparison Slide 85 85 Accounting for Baseline Risk Factors Age Gender Previous MI Smoking status Baseline LDL-C, HDL-C, TG Baseline DBP & SBP Additional analyses also included revascularization, diabetes and obesity Slide 86 86 Meta-Analysis Endpoints Considered Fatal or non-fatal MI Ischemic stroke Composite: CHD death, non-fatal MI, CABG, PTCA or ischemic stroke Slide 87 87 Model 1: Multivariate Cox proportional hazards model Patients combined across trials; trial effect is a fixed covariate Meta-Analysis Models H(t) = 0 (t)exp(Z + S + T ) Baseline Hazards constant Covariates Study effects Treatment Effects Slide 88 88 RRR = Relative Risk Reduction Relative Risk (95% CI) RRR Relative Risk Reduction Cox Proportional Hazards All Trials Prava+ASA vs ASA alone Prava+ASA vs Prava alone Fatal or Non-Fatal MI 0.4000.8001.0000.600 0.4000.8001.0000.600 CHD Death, Non-Fatal MI, CABG, PTCA, or Ischemic Stroke Prava+ASA vs ASA alone Prava+ASA vs Prava alone 24% 0.76 13% 0.87 31% 0.69 26% 0.74 Prava+ASA vs ASA alone Prava+ASA vs Prava alone 29% 0.71 31% 0.69 Ischemic Stroke 0.4000.8001.0000.600 Slide 89 89 Model 2: Same as Model 1 except Allows trial heterogeneity: Bayesian hierarchical (random effects) model of trial effect Meta-Analysis Models H(t) = 0 (t)exp(Z + S + T ) Baseline Hazards piecewise-constant Covariates Study effects Hierarchical Treatment Effects Slide 90 90 0.000 0.025 0.050 0.075 0.100 012345 Year Model 2 Hierarchical, Random Effects Fatal or Non-Fatal MI Placebo Prava alone ASA alone Prava+ASA Cumulative Proportion of Events Slide 91 91 0.000 0.005 0.010 0.015 0.020 0.025 012345 Model 2 Hierarchical, Random Effects Ischemic Stroke Only ASA alone Prava+ASA Year Cumulative Proportion of Events Prava alone Placebo Slide 92 92 0.00 0.05 0.10 0.15 0.20 0.25 012345 Year Model 2 Hierarchical, Random Effects CHD Death, Non-Fatal MI, CABG, PTCA, or Ischemic Stroke Prava+ASA ASA alone Prava alone Placebo Cumulative Proportion of Events Slide 93 93 Combination is More Effective than Either Agent Alone Pravastatin + aspirin provides benefit for all three endpoints: 24% - 34% RRR compared with aspirin 13% - 31% RRR compared with pravastatin This benefit was similar in Models 1 and 2 This benefit was consistent in both LIPID and CARE trials This benefit was similar in Models 1 and 2 This benefit was consistent in both LIPID and CARE trials Slide 94 94 Model 2: Fatal or Non-Fatal MI Cumulative Proportion of Events 0.000 0.025 0.050 0.075 0.100 Year 012345 Prava+ASA ASA alone Prava alone Placebo 0.000 0.005 0.010 0.015 0.025 Year 012345 0.020 HazardPrava+ASA ASA alone Prava alone Placebo Slide 95 95 Model 3: Same as Model 2 except Treatment hazard ratios vary over time Meta-Analysis Models H(t) = 0 (t)exp(Z + S ) Baseline Hazards piecewise-constant Within treatment Covariates Study Effects Hierarchical Slide 96 96 Model 3: Fatal or Non-Fatal MI Cumulative Proportion of Events 0.000 0.025 0.050 0.075 0.100 Year 012345 Prava+ASA ASA alone Prava alone Placebo 0.000 0.005 0.010 0.015 0.030 Year 5 Separate Analyses: One per Year 012345 0.020 Hazard 0.025 Prava+ASA ASA alone Prava alone Placebo Slide 97 97 Probability of synergy between pravastatin & aspirin EndpointModel 2Model 3 All events0.9830.985 Cardiac events0.9450.947 Any MI0.9110.923 Stroke0.9240.906 Death0.997 Slide 98 98 Conclusion of Hazard Analysis over Time Benefit of pravastatin+aspirin over aspirin was present in each year of the 5-year duration of the trials Benefit of pravastatin+aspirin over pravastatin was present in each year of the 5-year duration of the trials Benefits estimated from Model 1 (and confidence intervals) confirmed by more general models and fewer assumptions Benefit of pravastatin+aspirin over pravastatin was present in each year of the 5-year duration of the trials Benefits estimated from Model 1 (and confidence intervals) confirmed by more general models and fewer assumptions Slide 99 99 Hierarchical modeling in design Using historical information Combining results from multiple concurrent trials (or many centers) Slide 100 100 Hierarchical modeling & dose-response Example: drug Z (rozuvastatin) vs drug A (atorvastatin) (Berry et al., 2002, American Heart Journal) Slide 101 101 Studies involving drugs A and Z*, with %change from baseline. %Change Study n DoseMeanSD Y 1. 461027100.73 452034100.66 2. 451035.3 80.647 3. 14Placebo 1.4180.986 13 516.7170.833 162033.2180.668 128041.4180.586 4.2221035140.65 5.2102045.0100.55 2154051.1120.489 6.1321037130.63 7.133Placebo 1121.01 7071036130.64 8. 17Placebo 0 81.00 181035 80.65 9. 411035130.65 10. 731038100.62 512046 80.54 614051100.49 108054 90.46 Slide 102 102 Study n DoseMeanSD Y 11.541030180.70 12. 18971037.6NA0.624 13.12Placebo 7.6 91.076 11 2.525.0 90.75 13 529.0 90.71 111041.0 90.59 102044.3 90.557 114049.7 90.503 118061.0 90.39 14.401029120.71 15. 1648046NA0.54 16.12Placebo 5.1 8.11.051 151043.9 7.80.561 138056.9 8.30.431 14 1*35.9 7.70.641 15 2.5*40.6 9.90.594 16 5*44.1 8.30.559 1710*51.7 8.70.483 1720*55.512.80.445 1840*63.2 8.70.368 17.17Placebo 0.810.61.008 1540*61.9 7.20.381 3180*62.9 7.80.371 Slide 103 103 Dose-response model Y ij = exp{ s + a t + b t log(d)} + ij s for study t for drug d for dose i for observation (1,..., 43) j for patient within study/dose ij is N(0, 2 ) Priors dont matter much, except... Slide 104 104 Prior for s ~ N(0, 2 ) 2 is important 2 large means studies heterogeneous little borrowing 2 small means studies homogeneous much borrowing Prior of 2 is IG(10, 10) Prior mean and sd are 0.10 & 0.017 Slide 105 105 Likelihood Calculations of posterior & predictive distributions by MCMC Slide 106 106 Posterior means and SDs Parameter MeanStDev a P 0.00160.027 a A 0.0730.055 a Z 0.34 0.059 b A 0.1490.021 b Z 0.1460.019 0.1520.024 0.0870.011 Slide 107 107 Posterior means and SDs Par. MeanStDevPar. MeanStDev 1 0.1020.023 10 0.0720.022 2 0.0170.032 11 0.0520.027 3 0.0620.025 12 0.0540.013 4 0.0140.018 13 0.0280.024 5 0.0720.035 14 0.0630.031 6 0.0430.022 15 0.1040.042 7 0.0150.013 16 0.0700.031 8 0.0020.029 17 0.0170.033 9 0.0130.033 Slide 108 108 Model fit Slide 109 109 Interval estimates for pop. mean: model (line) vs standard (box) Slide 110 110 Study/dose- specific interval estimates: model (line) vs standard (box) Slide 111 111 Posterior distn of reduction (95% intervals) Drug A Drug Z Slide 112 112 Posterior distn of mean diff, A Z Slide 113 113 Really neat... Using predictive probabilities for designing future studies Contour plots Slide 114 114 Observed %Y for future study with n A =n Z =20 d A =d Z =10 Z A Slide 115 115 Observed %Y for future study with n A =n Z =100 d A =d Z =10 Z A Slide 116 116 Observed %Y for future study with n A =n Z =20 d A =10, d Z =5 Z A Slide 117 117 Observed %Y for future study with n A =n Z =100 d A =10, d Z =5 Z A Slide 118 118 STELLAR trial results (each n160) -50% -54% -58% -36% -41% -46% -52% Predictedatorva Predictedrosuva Slide 119 119 Posterior distn of reduction (95% intervals) Drug A Drug Z Recall: Slide 120 120 Adaptive Phase II: Finding the Best Dose Scott M. Berry scott@berryconsultants.com Scott M. Berry scott@berryconsultants.com Slide 121 121 Doses Standard Parallel Group Design Equal sample sizes at each of k doses. Slide 122 122 Response Doses True dose-response curve (unknown) Slide 123 123 Response Doses Observe responses (with error) at chosen doses Slide 124 124 Response Doses True ED 95 Dose at which 95% max effect Slide 125 125 Response Doses Uncertainty about ED95 ? Slide 126 126 Response Doses True ED 95 Solution: Increase number of doses Solution: Slide 127 127 Response Doses True ED 95 But, enormous sample size, and... wasted dose assignmentsalways! Slide 128 128 Solutions Lots of doses (continuum?) Adaptive Allocation Model dose response Define what you are looking for Stop when you find what you are looking for Yogi Berra-ism: If you dont know where you are going, how do you know when you get there? Slide 129 129 Dose Finding Trial Real example (all details hidden, but flavor is the same) Delayed Dichotomous Response (random waiting time) Combine multiple efficacy + safety in the dose finding decision Use utility approach for combining various goals Multiple statistical goals Adaptive stopping rules Slide 130 130 Adaptive Approach Slide 131 131 Statistical Model The statistical model captures all the uncertainty in the process. Capture data, quantities of interest, and forecast future data Be flexible, (non-monotone?) but capture prior information on model behavior. Invisible in the process Slide 132 132 Empirical Data Observe Y ij for subject i, outcome j Y ij = 1 if event, 0 otherwise j = 1 is type #1 efficacy response j = 2 is type #2 efficacy response j = 3 is minor safety event j = 4 is major safety event Slide 133 133 Efficacy Endpoints Let d be the dose P j (d) probability of event j, dose d. j (d) ~ N( j, 2 ) IG(2,2) N(2,1) N(1,1) G(1,1) Slide 134 134 Safety Endpoint Let d i be the dose for subject i P j (d) probability of safety j, dose d. N(-2,1) N(1,1) G(1,1) Slide 135 135 Utility Function Multiple Factors: Monetary Profile (value on market) FDA Success Safety Factors Utility is critical: Defines ED ? Slide 136 136 Utility Function U(d)=U 1 (P 1 )U 2 (P 3 )*U 3 (P 0,P 2 )*U 4 (P 4 ) Monetary FDA Approval Extra Safety P 0 is prob efficacy 2 success for d=0 Slide 137 137 Monetary Utility Slide 138 138 Slide 139 139 Slide 140 140 Slide 141 141 U 3 : FDA Success DSMB? Slide 142 142 Statistical + Utility Output E[U(d)] E[ j (d)], V[ j (d)] E[P j (d)], V[P j (d)] Pr[d j max U] Pr[P 2 (d) > P 0 ] Pr[ P 2 >> P 0 | 250/per arm) each d >> means statistical significance will be achieved Slide 143 143 Allocator Goals of Phase II study? Find best dose? Learn about best dose? Learn about whole curve? Learn the minimum effective dose? Allocator and decisions need to reflect this (if not through the utility function) Calculation can be an important issue! Slide 144 144 Allocator Find best dose? Learn about best dose? Find the V* for each dose ==> allocation probs d* is the max utility dose, d** second best Slide 145 145 Allocator V*(d0) = V*(d=0) = Slide 146 146 Allocator Drop any r d 147 Decisions Find best dose? Learn about best dose? Shut down allocator w j if stop!!!! Stop trial when both w j = 0 If Pr(P 2 (d*) >> P 0 ) < 0.10 stop for futility If found, stop: Pr(d = d*) > C 1 Pr(P 2 (d*) >> P 0 )>C 2 Slide 148 148 More Decisions? Ultimate: EU(dosing) > EU(stopping)? Wait until significance? Goal of this study? Roll in to phase III: set up to do this Utility and why? are critical and should be done--easy to ignore and say it is too hard. Slide 149 149 Simulations Subject level simulation Simulate 2/day first 70 days, then 4/day Delayed observation exponential with mean 10 days Allocate + Decision every week First 140 subjects 20/arm Slide 150 150 Scenario #1 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.250.100.050.0600 0.50.130.080.0700.063 10.170.120.0800.323 2.50.200.150.0900.457 50.230.180.1000.532 100.300.250.1100.656 Stopping Rules: C 1 = 0.80, C 2 = 0.90 MAX Slide 151 151 18 1 0 2 20 2 0 1 0 18 2 0 2 15 5 0 3 5 19 5 0 3 1 17 4 2 3 18 5 3 2 Slide 152 152 Dose Probabilities 0.25.512.5510 P(>>Pbo).18.33.27.29.67 P(max).01.04.06.04.33.52 P(2nd).03.06.10.13.35.32 Alloc.06.01.02.04.06.35.46 Slide 153 153 20 1 0 3 20 2 0 1 0 18 2 0 2 19 5 1 4 1 19 5 0 3 25 7 8 2 7 24 7 5 2 7 Slide 154 154 Dose Probabilities 0.25.512.5510 P(>>Pbo).12.38.36.38.92.91 P(max).00.02.04.41.53 P(2nd).00.03.06.07.47.37 Alloc.00.02.04.09.34.51 Slide 155 155 21 2 1 0 2 20 2 0 1 0 19 3 2 0 1 20 5 1 4 0 21 5 0 3 4 29 7 9 2 11 31 11 6 3 17 Slide 156 156 Dose Probabilities 0.25.512.5510 P(>>Pbo).13.39.38.26.97.85 P(max).00.02.03.01.39.55 P(2nd).00.03.10.05.46.35 Alloc.11.00.03.10.05.46.35 Slide 157 157 23 2 1 0 4 20 2 0 1 0 20 4 2 0 21 5 1 4 25 5 1 4 0 36 7 10 3 10 45 12 10 3 16 Slide 158 158 Dose Probabilities 0.25.512.5510 P(>>Pbo).16.41.38.48.93 P(max).00.02.03.04.26.65 P(2nd).00.05.07.10.49.29 Alloc.00.08.11.18.35.28 Slide 159 159 26 2 1 0 1 20 2 0 1 0 20 4 2 0 25 5 1 4 6 26 6 2 4 5 44 7 13 3 12 52 13 10 4 15 Slide 160 160 Dose Probabilities 0.25.512.5510 P(>>Pbo).16.40.31.41.98.89 P(max).00.02.03.06.27.63 P(2nd).00.06.12.48.28 Alloc.16.00.10.04.13.26.30 Slide 161 161 26 2 1 0 6 20 2 0 1 0 21 4 2 0 3 26 6 1 4 5 33 7 3 4 5 52 8 13 4 10 61 18 15 4 12 Slide 162 162 Dose Probabilities 0.25.512.5510 P(>>Pbo).13.36.32.65.96 P(max).00.01.09.08.81 P(2nd).00.05.23.52.15 Alloc Slide 163 163 Trial Ends P(10-Dose max Util dose) = 0.907 P(10-Dose >> Pbo 250/arm) = 0.949 280 subjects: 32, 20, 24, 31, 38, 62, 73 per arm Slide 164 164 Operating Characteristics Pbo0.250.512.5510 SS392125376389110 Pmax---0.00 0.040.96 SS66 Pmax---0.00 0.010.060.93 Slide 165 165 Operating Characteristics AdaptiveConstant P(Success)0.9360.810 P(Cap)0.0640.190 P(Futility)0.000 Mean SS384459 SD SS186224 Mean TDose17541263 Max TDose48182370 Slide 166 166 Scenario #2 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.250.100.050.0600 0.50.130.080.0700.063 10.170.120.0800.323 2.50.200.150.1000.452 50.230.180.1500.502 100.250.200.4000.302 Stopping Rules: C 1 = 0.80, C 2 = 0.90 Slide 167 167 Operating Characteristics Pbo0.250.512.5510 SS71274181137172164 Pmax---0.00 0.030.220.600.16 SS100 Pmax---0.00 0.030.200.440.33 Slide 168 168 Operating Characteristics AdaptiveConstant P(Success)0.3140.266 P(Cap)0.6860.734 P(Futility)0.000 Mean SS694702 SD SS193190 Mean TDose29541937 Max TDose44892455.25 Slide 169 169 Simulation #3 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.10.100.050.0600 0.50.130.080.0700.063 10.300.250.1100.656 2.50.170.120.0800.323 50.200.150.0900.457 100.230.180.1000.532 Stopping Rules: C 1 = 0.80, C 2 = 0.90 Slide 170 170 Operating Characteristics Pbo0.250.512.5510 SS5323281195276102 Pmax---0.00 0.920.000.010.07 SS87 Pmax---0.00 0.830.000.020.15 Slide 171 171 Operating Characteristics AdaptiveConstant P(Success)0.9060.596 P(Cap)0.0920.404 P(Futility)0.0020.000 Mean SS453606 SD SS187205 Mean TDose16631662 Max TDose37712384.25 Slide 172 172 Scenario #4 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.250.060.05 00 0.50.060.05 00 10.060.05 00 2.50.250.200.1000.573 50.250.200.1000.573 100.250.200.1000.573 Stopping Rules: C 1 = 0.80, C 2 = 0.90 Slide 173 173 Operating Characteristics Pbo0.250.512.5510 SS53212223150160163 Pmax---0.00 0.270.320.40 SS92 Pmax---0.00 0.280.330.40 Slide 174 174 Operating Characteristics AdaptiveConstant P(Success)0.5140.408 P(Cap)0.4860.592 P(Futility)0.000 Mean SS591647 SD SS239220 Mean TDose28401780 Max TDose48152448.25 Slide 175 175 Scenario #5 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.10.070.06 00 0.50.080.07 00 10.090.08 00 2.50.090.080.0900 5 0.080.1000 100.090.080.1100 Stopping Rules: C 1 = 0.80, C 2 = 0.90 Slide 176 176 Operating Characteristics Pbo0.250.512.5510 SS92917566768390 Pmax---0.450.040.070.100.130.21 SS84 Pmax---0.440.040.080.120.150.17 Slide 177 177 Operating Characteristics AdaptiveConstant P(Success)0.0040.006 P(Cap)0.4840.544 P(Futility)0.5120.450 Mean SS574589 SD SS250258 Mean TDose16371615 Max TDose3223.52523.75 Slide 178 178 Scenario #6 DoseP1P1 P2P2 P3P3 P4P4 UTIL 00.060.05 00 0.10.060.05 00 0.50.060.05 00 10.060.05 00 2.50.060.05 00 50.060.05 00 Stopping Rules: C 1 = 0.80, C 2 = 0.90 Slide 179 179 Operating Characteristics Pbo0.250.512.5510 SS66775134384143 Pmax---0.900.010.02 0.03 SS56 Pmax---0.860.010.020.03 0.05 Slide 180 180 Operating Characteristics AdaptiveConstant P(Success)0.000 P(Cap)0.1220.190 P(Futility)0.8780.810 Mean SS350395 SD SS215241 Mean TDose8111086 Max TDose24042428.75 Slide 181 181 Bells & Whistles Interest in Quantiles Minimum Effective Dose Significance, control type I error Seamless phase II --> III Partial Interim Information Biomarkers of endpoint Continuous (& Poisson) Continuum of doses (IV)--little additional n!!! Slide 182 182 Conclusions Approach, not answers or details! Shorter, smaller, stronger! Better for company, FDA, Science, PATIENTS Why study?--adaptive can help multiple needs. Adaptive Stopping Bid Step!