Gatekeeping Testing Strategies in Clinical Trials Alex Dmitrienko, Ph.D. Eli Lilly and Company...
-
Upload
angelina-carney -
Category
Documents
-
view
221 -
download
1
Transcript of Gatekeeping Testing Strategies in Clinical Trials Alex Dmitrienko, Ph.D. Eli Lilly and Company...
Gatekeeping Testing Strategies in Clinical Trials
Alex Dmitrienko, Ph.D.Eli Lilly and Company
FDA/Industry Statistics WorkshopSeptember 2004
Slide 2
Outline
Gatekeeping strategies• account for the hierarchical structure of multiple analyses
(comparisons) and preserve the overall false positive rate
Clinical trial examples• registration trials with multiple primary and secondary
endpoints (trials in patients with depression and acute lung injury)
• dose-finding trials (trial in patients with hypertension)
Slide 3
Primary versus secondary findings
Findings with respect to secondary endpoints provide much useful information • useful to prescribing physicians and patients
FDA guidance “Clinical studies section of labeling for prescription drugs and biologics”• “The CLINICAL STUDIES section should present those
endpoints that are essential to establishing the effectiveness of the drug (or that show the limitations of effectiveness) and those that provide additional useful and valid information about the activities of the drug”
Slide 4
Primary versus secondary findings
Dilemma• regulatory agencies and pharmaceutical companies have
long debated what secondary findings should be included in the product label
• regulatory agencies are concerned that pharmaceutical companies tend to present favorable data and ignore unfavorable data
Gatekeeper strategies offer one potential solution to the dilemma
Slide 5
Types of gatekeeping strategies
Two types of gatekeeping strategies• sequential strategies have been used for 10 years• parallel strategies were introduced by Dmitrienko, Offen
and Westfall (2003)
Introduce general gatekeeping framework• based on the closed testing principle (Marcus et al, 1976)• focus on strategies derived using Bonferroni’s test• easily extended to more powerful tests that account for
the correlation among the endpoints (Dunnett’s test, resampling tests)
Slide 6
Example 1: One primary endpoint
Depression trial• Experimental drug is compared to placebo• Single primary endpoint
– 17-item Hamilton depression rating scale (HAMD 17 score)
• Trial is declared successful if the drug is superior to placebo
• Two important secondary endpoints– response rate based on the HAMD 17 score
– remission rate based on the HAMD 17 score
• Can the secondary findings be included in the product label?
Slide 7
Sequential gatekeeping strategy
Step 1: Perform the primary analysis
Step 2: Perform the secondary analyses with an adjustment for multiplicity if the primary analysis yielded a significant result
A1: HAMD17A2: Response rate
A3: Remission rate
Family 2Family 1 (gatekeeper)
Slide 8
Sequential gatekeeping strategy
Endpoint Raw p Adjusted p Primary: HAMD 17 0.046 0.046 Secondary: Response rate 0.048 0.048 Secondary: Remission rate 0.021 0.042
Primary analysis: No adjustment for multiplicitySecondary analyses: Stepwise Holm’s test
All primary and secondary findings are significant at 5% level
Slide 9
Example 2: Multiple primary endpoints
Clinical trial in patients with acute lung injury• Experimental drug is compared to placebo• Two primary endpoints
– number of days patients are off mechanical ventilation (vent-free days)
– 28-day all-cause mortality rate
• Trial is declared successful if the drug is superior to placebo with respect to either endpoint
• Two important secondary endpoints– number of days patients are out of ICU (ICU-free days)
– overall quality of life at the end of the study
• Can the secondary findings be included in the product label?
Slide 10
Parallel gatekeeping strategy
Step 1: Perform the primary analyses with an adjustment for multiplicity
Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result
A1: Vent-free days
A2: Mortality
A3: ICU-free days
A4: Quality of life
Family 1 (gatekeeper)
Family 2
Slide 11
Parallel gatekeeping strategy
Step 1: Perform the primary analyses with an adjustment for multiplicity
Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result
A1: Vent-free days
A2: Mortality
A3: ICU-free days
A4: Quality of life
Family 1 (gatekeeper)
Family 2
Slide 12
Parallel gatekeeping strategy
Step 1: Perform the primary analyses with an adjustment for multiplicity
Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result
A1: Vent-free days
A2: Mortality
A3: ICU-free days
A4: Quality of life
Family 1 (gatekeeper)
Family 2
Slide 13
Statistical details
Sequential families of null hypotheses• Null hypotheses H11, H12, H21, H22 grouped into two
families F1={H11, H12} and F2 ={H21, H22}
Parallel gatekeeping testing procedure1. Overall Type I error rate is no greater than (e.g., 0.05)
2. The null hypotheses in F2 can be tested if at least one hypothesis in F1 has been rejected [parallel testing]
3. Adjusted p-values for the hypotheses in F1 do not depend on the p-values associated with the hypotheses in F2 [primary analyses cannot depend on secondary analyses]
Slide 14
Closed testing
Closed family of hypotheses• Consider all 15 intersections of the null hypotheses
– {H11 or H12 or H21 or H22}, {H11 or H12 or H21}, etc
• Specify a test for each intersection hypothesis that controls the Type I error rate, e.g., Bonferroni test
• Establish implication relationships– Intersection hypothesis {H11 or H12} implies H11 and H12, etc
• Tests for original hypotheses– Reject a null hypothesis if all intersection hypotheses implying it have
been rejected
Slide 15
Decision matrixIntersection hypothesis Rejection rule H11 or H12 or H21 or H22 Z11>z/2 or Z12>z/2 H11 or H12 or H21 Z11>z/2 or Z12>z/2 H11 or H12 or H22 Z11>z/2 or Z12>z/2 H11 or H12 Z11>z/2 or Z12>z/2 H11 or H21 or H22 Z11>z/2 or Z21>z/4 or Z22>z/4 H11 or H21 Z11>z/2 or Z21>z/2 H11 or H22 Z11>z/2 or Z22>z/2 H11 Z11>z/2 H12 or H21 or H22 Z12>z/2 or Z21>z/4 or Z22>z/4 H12 or H21 Z12>z/2 or Z21>z/2 H12 or H22 Z12>z/2 or Z22>z/2 H12 Z12>z/2 H21 or H22 Z21>z/2 or Z22>z/2 H21 Z21>z H22 Z22>z
Slide 16
Parallel gatekeeping strategy
Two scenarios• significant improvement in the mean number of ventilator-
free days and 28-day all-cause mortality• significant improvement in 28-day all-cause mortality but
not in mean number of ventilator-free day
Weighted analyses• primary endpoints are unequally weighted to reflect their
relative importance (likelihood of success)– weight=0.9 for number of ventilator-free days
– weight=0.1 for 28-day all-cause mortality
Slide 17
Parallel gatekeeping strategy
Scenario 1• significant improvement in the mean number of ventilator-
free days and 28-day all-cause mortality
Endpoint Raw p Adjusted p Primary: Vent-free days 0.024 0.027 Primary: Mortality 0.003 0.030 Secondary: ICU-free days 0.026 0.029 Secondary: Quality of life 0.002 0.027
Primary analyses: Bonferroni’s testSecondary analyses: Stepwise Holm’s test
All primary and secondary findings are significant at 5% level
Slide 18
Parallel gatekeeping strategy
Scenario 2• significant improvement in 28-day all-cause mortality but
not in mean number of ventilator-free day
Endpoint Raw p Adjusted p Primary: Vent-free days 0.084 0.093 Primary: Mortality 0.003 0.030 Secondary: ICU-free days 0.026 0.093 Secondary: Quality of life 0.002 0.040
Primary analyses: Bonferroni’s testSecondary analyses: Stepwise Holm’s test
The primary mortality analysis and secondary quality of life analysis are significant at 5% level
Slide 19
Example 3: Dose-finding study
Clinical trial in patients with hypertension• Four doses of an experimental drug are compared to
placebo– doses are labeled as D1, D2, D3 and D4
• Primary endpoint– reduction in diastolic blood pressure
• Objectives of the study– find the doses with a significant reduction in diastolic blood pressure
compared to placebo
– study the shape of the dose-response curve
Slide 20
Example 3: Dose-finding study
Step 1: Compare doses D3 and D4 to placebo
Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
Step 3: Perform various pairwise dose comparisons if at least one comparison at Step 2 is significant
A1: D4 vs. P
A2: D3 vs. P
A3: D2 vs. P
A4: D1 vs. P
Pairwise comparisons
Family 1 (gatekeeper)
Family 2 (gatekeeper)
Family 3
Slide 21
Example 3: Dose-finding study
Step 1: Compare doses D3 and D4 to placebo
Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
Step 3: Perform various pairwise dose comparisons if at least one comparison at Step 2 is significant
A1: D4 vs. P
A2: D3 vs. P
A3: D2 vs. P
A4: D1 vs. P
Pairwise comparisons
Family 1 (gatekeeper)
Family 2 (gatekeeper)
Family 3
Slide 22
Example 3: Dose-finding study
Step 1: Compare doses D3 and D4 to placebo
Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
Step 3: Perform various pairwise dose comparisons if at least one comparison at Step 2 is significant
A1: D4 vs. P
A2: D3 vs. P
A3: D2 vs. P
A4: D1 vs. P
Pairwise comparisons
Family 1 (gatekeeper)
Family 2 (gatekeeper)
Family 3
Slide 23
Example 3: Dose-finding studyM
ean
redu
ctio
n in
DB
P (m
mH
g)
0
5
10
Dose0 1 2 3 4
Slide 24
Parallel gatekeeping strategyComparison Raw p Adjusted p
Gatekeeping Holm Dunnett
procedure procedure procedure
D4 vs. P 0.0008 0.0016 0.0055 0.0030
D3 vs. P 0.0135 0.0269 0.0673 0.0459
D2 vs. P 0.0197 0.0394 0.0787 0.0656
D1 vs. P 0.7237 1.0000 1.0000 0.9899
D4 vs. D1 0.0003 0.0394 0.0021
D4 vs. D2 0.2779 1.0000 0.8338
D3 vs. D1 0.0054 0.0394 0.0324
D3 vs. D2 0.8473 1.0000 1.0000
Doses D2, D3 and D4 are significantly different from placebo at 5% level
Slide 25
Summary
Gatekeeping strategies can be successfully used in• pivotal trials with multiple primary and secondary
endpoints • dose-finding studies
Registration trials• a priori designation of gatekeeping strategy allows
additional data useful to physician and patient to be presented in the product label
Dose-finding studies• efficient tests of dose-response relationship
Slide 26
Extensions
More powerful gatekeeping tests• based on more powerful tests, e.g., Simes test• based on tests accounting for the correlation among the
endpoints (exact parametric tests such as Dunnett’s test and approximate resampling-based Westfall-Young tests)
Software implementation• SAS programs for gatekeeping tests can be found in
Dmitrienko, Molenberghs, Chuang-Stein, Offen. (2004). Analysis of Clinical Trials: A Practical Guide. SAS Publishing, Cary, NC.
Slide 27
References
Papers• Dmitrienko, Offen, Westfall. (2003). Gatekeeping
strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 22, 2387-2400.
• Marcus R, Peritz E, Gabriel KR. (1976). On closed testing procedure with special reference to ordered analysis of variance. Biometrika. 63, 655-660.
• Westfall, Krishen. (2001). Optimally weighted, fixed sequence, and gatekeeping multiple testing procedures. Journal of Statistical Planning and Inference. 99, 25-40.