Designs meriting special considerationcsph.ucdenver.edu/sites/kittelson/Bios6649-2013/L... ·...

1

Designs meriting special consideration

Non-inferiority designs

Regulatory guidance (FDA)

Example 1: DaptomycinI Fixed-sample designI Trial monitoringI Evaluation

Example 2: Rivaroxaban in non-valvular atrial fibrillation

7. Non-inferiority Trials 7. Non-inferiority Trials 29 Apr 2013 1 / 42

2

Designs meriting special consideration


Recall reasons for early termination (lecture 4.3)

Why would you want to stop a study early?I Superiority study:

F For superiority (reject H0 : θ ≤ 0)F For lack of superiority (reject HA : θ ≥ θ+)

I Approximate equivalence study:F For lack of inferiority (reject H0 : θ ≤ θ−)F For lack of superiority (reject HA : θ ≥ θ+)

I Non-inferiority study:F For lack of inferiority (reject H0 : θ ≤ θ−)F For inferiority (reject HA : θ ≥ 0)

I Equivalence (2-sided) study:F For superiority (reject θ ≤ 0)F For inferiority (reject θ ≥ 0)F For both non-inferiority and non-superiority (reject both θ ≤ θ−

and θ ≥ θ+)


3

FDA Regulatory Guidance



4


III. General Consideration of NI studies:

A. Basic principles of a NI study

1. Superiority vs NI2. Logic of NI trial3. Reasons for using NI design4. The non-inferiority margin5. Assay sensitivity and choosing NI margin6. Regulatory conclusions

B. Practical Considerations for use of NI designs

1. Consider alternative designs2. Number of studies needed3. Statistical inferences4. Choice of active control5. Choice of NI method


5


A. Basic principles

A. Basic principles of a NI study1. Superiority vs NI

F Superiority study: Interpretable in their own right ... results speakfor themselves without additional assumption.

F Non-inferiority study: Requires knowing something that is notmeasured in the study; that is:

The active control had its expected effect in the NI study.

Assay sensitivity (think power):F Trial could have distinguished an effective from an ineffective drug.F Establishing NI is meaningless without assay sensitivity.F Knowing if the trial has assay sensitivity relies on external

information.


6

III. General Consideration of NI studies:2. Logic of the NI trial

I Superiority study: establish significant benefit over placebo:


7

III. General Consideration of NI studies:2. Logic of the NI trial

I Non-inferiority study: benefit at least as large as control superiority:


8


2. Logic of the NI trialI Non-inferiority study: Choice of non-inferiority margin (M1)

(1) Treatment effect based on historical experience with the activecontrol drug

(2) Assessment of the likelihood that the current effect of the activecontrol is similar to the past effect (the constancy assumption)

(3) Assessment of the quality of the NI trial, particularly looking fordefects that could produce bias toward the null.[Note: for this reason the size of M1 cannot usually be assessed untilthe trial is finished.]


9


“Constancy Assumption”I Historic evidence of sensitivity to drug effects (HESDE) (ICH E-10)

F HESDE means that past trials that used a specific active treatmentregularly showed this treatment to be superior to placebo.

I The conclusion that HESDE applies only when the NI study issufficiently similar to the past studies:

F Characteristics of the patient population.F Concomitant treatments.F Definitions and ascertainment of study endpointsF Dose of active control, entry criteria, analytic approaches.


10


Summary commentsI Beware of ‘bias toward the null’ in a non-inferiority trial:

F Inflated variance (⇒ poor assay sensitivity):Outcome ascertainmentAdequate blinding (central adjudication)Compliance

F Missing data!!

I “Biocreep” or “Noninferiority creep”:F Repeated comparisons to latest newest treatment results in steady

drift to inferiority relative to placebo.

I Avoid NI trials whenever possible.F “If it is ethical to conduct a placebo-controlled trial, then it is

unethical not to” (Lloyd Fisher).


11

Non-inferiority trials

Two examples:I Daptomycin for Staph infection

F Trial results: Fowler, VG, et.al. NEJM (2006); 355(7):653-65F Editorial: Grayson, ML. NEJM (2006); 355(7):724

I Rivaroxaban for atrial fibrillationF Trial results: Patel, MR, et.al. NEJM (2011); 365: 883-91F Editorial: Fleming, TR, Emerson SS. NEJM (2011); 365(17): 1557


12


Example: Daptomycin NEJM (2006); 355(7):653-65

Structuring parameter space:

Background: Patients with bacteremia and endocarditis caused byS. aureus need alternative antibiotic therapies. Current antibiotictreatments are successful in about 65% of the cases.

Clinical question: Can daptomycin be used as another approachfor treating S. aureus bacteremia?

Study interventions: open label assignment toI Standard antibiotic careI Daptomycin

Primary outcome: Clinical success at 42 days (failure = noresponse to drug based on ongoing signs and symptoms)

Functional: θ1, θ0: population success probability withdaptomycin and standard care

Contrast: θ = θ1 − θ07. Non-inferiority Trials 7. Non-inferiority Trials 29 Apr 2013 12 / 42

13



Trial information:

Non-inferiority definition: “Lower bound of the 95% confidenceintervals must be within the prespecified noninferiority margin of20 percent and the upper bounds containing 0 percent.”

Sample size: 90 patients per group gives 80% power for NI ifθ0 = θ1 = 0.65.

Sample size evaluation: If θ0 = θ1 = 0.65, then the 95% CI hashalf-width:

1.96

√(0.65× 0.35)

2

90= 0.14


14

Structuring parameter space: Non-inferiority designsWhat should we decide with an infinite sample size (θ is known)?

H

G

F

E

D

C

B

A

Pot

entia

l tria

l res

ults

(in

finite

sam

ple

size

)

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−

Inferiority Superiority

ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+


15

Structuring parameter space: Non-inferiority designs


What should we decide with an infinite sample size(θ known)?

A, B, C, D ⇒ Superior (use daptomycin)

E ⇒ Unimportant inferiority (use daptomycin?)

F, G, H ⇒ Inferior (do not use daptomycin)

May be willing to accept some inferiority:I Diversity of antibiotics may avoid evolution of resistant strainsI Unresponsive patients may respond to a different treatment


16



Other considerationsI In the marginal situations the acceptability of daptomycin would

also consider:

* The clinical “importance” of a difference, which is difficult toquantify.

* Nature of the primary endpoint (e.g., survival vs treatment success).* Effects on secondary endpoints (toxicity, rescue therapies)?* Potential for long-term effects that could not be measured in this

trial.

I Bottom line:

* There are grey-areas in parameter space.* These will affect the design (e.g., particularly in non-inferiority

trials).

I Remark: A question that cannot be answered with an infiniteamount of data cannot be answered with a finite amount of data.


17

Structuring parameter space: Non-inferiority designsWhat should we decide with a finite sample size(θ is estimated with uncertainty)?

( )

( )

( )

( )

( )

( )

( )

( )

H

G

F

E

D

C

B

A

Pot

entia

l tria

l res

ults

(co

nfid

ence

inte

rval

s)

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−


ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+


18



What should we decide with an infinite sample size(θ known)?

A, B, C, D ⇒ Superior (use daptomycin)

E, F ⇒ Unimportant inferiority (use daptomycin?)

G, H ⇒ Significant inferiority (do not use daptomycin)


19


Superiority/non-superiority studies

Application: Evaluating new therapies to determine if theyincrease benefits.

Defining Hypotheses:I Inferiority: θ ≤ θ∅

(Decide for superiority if inferiority is rejected)I Clinical Superiority: θ ≥ θ+

(Decide against superiority if clinical superiority is rejected)

Example: Suppose we hypothesize daptomycin is superior tostandard care and we want to discriminate between θ ≤ θ∅ andθ ≥ θ+:

Inferiority : θ ≤ θ∅ = 0.0

Clinical superiority : θ ≥ θ+ = 0.28


20


Non-inferiority/inferiority studies

Applications:I Evaluating new therapies to determine if they are non-inferior to

existing therapy.I Evaluating existing therapies to determine if they are harmful.

Defining Hypotheses:I Clinical Inferiority: θ ≤ θ−

(Decide for non-inferiority if clinical inferiority is rejected)I Superiority: θ ≥ θ∅

(Decide against non-inferiority if superiority is rejected)

Example: Suppose daptomycin is in routine use, but there is someevidence it may be harmful, and we want to discriminate betweenθ ≤ θ− and θ ≥ θ∅:

Clinical Inferiority : θ ≤ θ− = −0.28

Superiority : θ ≥ θ∅ = 0.0


21


Equivalence studies

Application:I Evaluating therapies to determine if they can be used

interchangeably.I (If not, then to select the best therapy.)

Defining Hypotheses:I Clinical inferiority (θ ≤ θ−)I Clinical superiority (θ ≥ θ+)I Equality (θ = θ∅)I Decisions

Decide equivalence if reject both clinical inferiority and superiority.Decide treatment A better than B if reject inferiority (of A).Decide treatment B better than A if reject superiority (of A).

Example: Daptomycin vs standard therapy:

Clinical Inferiority (of daptomycin): θ ≤ −0.28Clinical Superiority (of daptomycin): θ ≥ 0.28Equality: θ = 0


22


Types of equivalence (iroically: not all equivalents are the same)

Categories of equivalence studies include:I Bioequivalence: showing that two formulations of the same drug are

equivalent.

Reject θ ≥ θ∅ ⇒ Conclude inferior

Reject θ ≤ θ∅ ⇒ Conclude superior

Reject θ ≤ θ− and θ ≥ θ+ ⇒ Conclude equivalence

In some settings FDA defines bioequivalence as 0.8 < θ1/θ0 < 1.25.


23


Types of equivalence (iroically: not all equivalents are the same)

Categories of equivalence studies include:I Non-Inferiority:

Reject θ ≥ θ∅ ⇒ Conclude inferior

Reject θ ≤ θ− ⇒ Conclude non-inferior

I Approximate equivalence: Require that observed effect be beneficialand important inferiority is rejected:

Observe θ̂ > θ∅ ⇒ Conclude non-inferior

Reject θ ≤ θ− ⇒ Conclude non-inferior


24


Non-inferiority trials of new therapeutics:public health implications

Notes (see figure below):I Non-inferiority creep:

F Suppose treatment A replaces an existing treatment in a strictnon-inferiority design (as above).

The point estimate may indicate inferior (θ̂ < θ∅)F Now suppose treatment B replaces treatment A in a strict

non-inferiority design.The point estimate may indicate inferior (θ̂ < θ∅)

F If this process continues, there may be a steady creep that results inincreasingly inferior treatments.

F Confidence interval A on the following figure illustrates how aninferior point estimate could produce a non-inferior decision.


25



Notes (see figure below):I Approximate equivalence designs:

F Requiring a positive point estimate (confidence interval B) wouldavoid non-inferiority creep.

F Requiring a positive point estimate and requiring rejection ofθ ≤ θ− would allow a smaller sample size (confidence interval C).

F Note: Sample size may be relaxed somewhat, but not as much asthe difference illustrated in confidence intervals B and C.

F Note: Requiring a positive point estimate means that there is only50% power for a treatment that is truly equivalent (θ = θ∅).


26



Daptomycin example (see figure below)I Approximate equivalence design (confidence interval B)

F Non-inferiority bound = −14%.F Sample size: N = 180 (90 per group)

⇒ critical value = 0%.

I Reported trial design (confidence interval D)F Non-inferiority bound = −20%.F Sample size: N = 180 (90 per group)

⇒ critical value = −6%.

I Actual trial results (confidence interval E)F N = 235 (N = 120 daptomycin; N = 115 standard care)F Point estimate: 2.4%F CI: -10.2% to 15.1%


27

Structuring parameter space: Non-inferiority designsPotential ways to specify non-inferiority decisions

( )

( )

( )

( )

( )

C

B

A

Pot

entia

l tria

l res

ults

E

D

Dap

tom

ycin

tria

l

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−


ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+


28


Bottom line

Design of equivalence trials (including non-inferiority trials)requires careful consideration of the standards that define thedecision.

Poorly run equivalence trials can introduce bias toward the nullhypothesis (an equivalence conclusion).(See editorial with daptomycin paper.)


29

Monitoring plan for non-inferiority trial


Comparison of designs (illustrated using daptomycin):I Two-sided bioequivalence designI Shifted superiority designI Approximate equivalence designI Approximate equivalence with a single boundary


30


A not-so-hypothetical example

Stage 1: Investigator discovers bioequivalence

Investigator hypothesizes that daptomycin might work for sepsis:I “Knows” it cannot be superior, so wants to show it is equivalent to

standard careI Finds a website about bioequivalence and when he plugged in his

numbers it told him:

* Use 90 patients per group and measure the difference in successproportions.

* Conclude equivalence if the 95% CI for the difference in proportionsis contained entirely between -0.28 and 0.28.

A research assistant says he should really use a non-inferioritydesign.

A clinical trialist says he should worry about non-inferiority creep.

Investigator is seeking help from the biostatisticians - everyone isconfused ...


31



Stage 2: The research assistant says...

A non-inferiority website says the design has 80% power forequivalence.

You end up having to explain/compare these approaches, andbegin with the fixed-sample design:

I 2-sided bioequivalence design(dap.2side)

I 1-sided superiority design(dap.fixSup)

I 1-sided shifted superiority design (eventually you reproduce theirdesign)(dap.fixShift)

I 1-sided “approximate equivalence” design(dap.fixNI)


32



Stage 3: Enter the biostatistician

dap.2side <- seqDesign(prob.model=’proportions’, arms=2,

null.hypothesis=c(0.65,0.65), alt.hypothesis="calculate",

test.type=’two.sided’, variance="null", alpha=0.025,

sample.size=180, power=0.975, nbr.analyses=1)

dap.fixSup <- seqDesign(prob.model=’proportions’, arms=2,

null.hypothesis=c(0.65,0.65), alt.hypothesis="calculate",

test.type=’greater’, variance="null", alpha=0.025, sample.size=180,

power=0.975, nbr.analyses=1)

e <- 0.715

dap.fixShift <- update(dap.fixSup,epsilon=c(e,1-e))

dap.fixNI <- update(dap.fixSup,epsilon=c(0.5,0.5))


33


Design critical values (fixed sample designs):

Comparison of decision criteria (180 patients):

DesignName aJ bJ cJ dJ

(dap.2side) -0.14 -0.14 0.14 0.14(dap.fixSup) 0.14 NA NA 0.14

(dap.fixShift) -0.06 NA NA 0.06(dap.fixNI) 0 NA NA 0

Inference:95% CI have width ±0.14 centered at the critical value


34


Power curves (fixed-sample designs:

−0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4

0.0

0.2

0.4

0.6

0.8

1.0

Difference in Proportions

Pow

er (

Upp

er)

dap.fixSupdap.fixShift

dap.fixNI

(dap.2side and dap.fixSup have same upper power curve)


35



Stage 4: Study section requests interim analyses

Add interim analyses:

dap.2sideIA <- update(dap.2side,sample.size=c(60,120,180))

dap.SupIA <- update(dap.fixSup,sample.size=c(60,120,180))

dap.ShiftIA <- update(dap.fixShift,sample.size=c(60,120,180))

dap.NI.IA <- update(dap.fixNI,sample.size=c(60,120,180))


36



Stage 5: Focus switches to inferiority decision

Ethics demands rapid termination for inferiority

Increasing sensitivity of lower (aj) boundary

dap.NI.IAasym<- update(dap.fixNI,sample.size=c(60,120,180),

P=c(0.8,Inf,Inf,1))

Now the steering committee decides it is only necessary to stop forinferiority

dap.NI.IAinf<- update(dap.fixNI,sample.size=c(60,120,180),

P=c(0.8,Inf,Inf,Inf))


37



Stage 6: Evaluate design properties

Stopping boundaries, power, ASN, stopping probabilities,inference at boundaries

seqPlotBoundary(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,

fixed=F,col=c(1,2,4))

seqPlotPower(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,

fixed=F,col=c(1,2,4))

seqPlotASN(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,

fixed=F,col=c(1,2,4),prob=NULL)


38

Monitoring plan for non-inferiority trialProperties: stopping boundaries

0 50 100 150

−0.

4−

0.2

0.0

0.2

0.4

0.6

Sample Size

Diff

eren

ce in

Pro

port

ions

● dap.NI.IA● dap.NI.IAasym

● dap.NI.IAinf

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●


39


Properties: power

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

0.0

0.2

0.4

0.6

0.8

1.0


Pow

er (

Upp

er)

dap.NI.IAdap.NI.IAasym

dap.NI.IAinf


40


Properties: ASN

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

100

120

140

160

180


Sam

ple

Siz

e

Average Sample Size

dap.NI.IAdap.NI.IAasymdap.NI.IAinf


41

Example 2: Rivaroxaban

Properties: ASN

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

100

120

140

160

180


Sam

ple

Siz

e

Average Sample Size

dap.NI.IAdap.NI.IAasymdap.NI.IAinf


42

Summary Remarks: Non-inferiority TrialsGeneral principles in non-inferiority trials

I Assay Sensitivity: Absence of a significant difference does not implyno difference.

I Constancy assumption:* The effect of the comparator relative to placebo remains constant

across all trials* May break down because of changes other aspects such as:

- ancillary care- diagnostic methods and patient populations

I Non-inferiority is difficult to objectively define* The “non-inferiority bound”:

- depends on the setting (population, treatment, endpoints)- may be driven by cost/feasibility as opposed to public health

I Sloppy science may bias toward non-inferiority* Assure compliance* Endpoints must be clinically important/relevant* Avoid surrogates

Design principlesI Designs should consider potential inference (CI) rather than power.I Monitoring designs should evaluate inference on the boundaries.


Designs meriting special considerationcsph.ucdenver.edu/sites/kittelson/Bios6649-2013/L... ·...

Documents

Transcript of Designs meriting special considerationcsph.ucdenver.edu/sites/kittelson/Bios6649-2013/L... ·...