Noninferiority TrialsAn Overview

Hollie Sturgeon, PharmD

What is a noninferiority trial?A clinical trial wherein the objective is to establish that

the experimental treatment is not clinically worse than

the active comparison treatment by more than a

small, predetermined margin.

Treatment A$35/Month

Treatment B$50/Month

Similar to classically designed head-to-head superiority trials, noninferiority trials provide comparison

data for evaluating treatment alternatives.

CONCEPT: If Treatment A is similar enough in efficacy to Treatment B that the

difference between them is clinically negligible, then my patient can use the less

expensive alternative.

How are noninferiority trials different from other trials?

  Superiority Equivalence Noninferiority

ObjectiveTo determine if one

treatment is superior to another

To determine if an experimental treatment and an active reference treatment are similar in

effect within a prestated range

To determine that an experimental treatment

is not clinically worse than an active

reference intervention by more than a small,

preset margin

Null Hypothesis

There is no difference

between the two interventions.

One treatment is superior to the other

The experimental treatment is equivalent

to the comparator within the prestated

margin

Alternate Hypothesis

One treatment is superior to the

other

One treatment is superior to the other

The experimental treatment is no worse than the comparator

with respect to the prestated margin

Limit P (-∆ to +∆) (0 to +∆)

How common are noninferiority trials?

During the 10-year period between 1999 & 2009,

582 noninferiority trials were published.

Between 1989 and 1999, there was one.

Of the 43 New Drug Applications approved by

the FDA between 2002 and 2009, 2/3 cited

evidence obtained from noninferiority trials

“Our study identified a clear trend of increasing frequency of publication of noninferiority trials”[5]

Advantages of noninferiority trials

Useful when placebo control is inappropriate.

Not limited to pharmaceutical therapy

Can be used for risk-benefit analyses

Appropriate for comparing a specific intervention to

itself (dose vs. dose or formulation vs. formulation)

Provides evidence for inferiority, noninferiority OR

superiority claims

Disadvantages of noninferiority trials Must meet specific design and analysis parameters

to be useful.

These requirements appear to be poorly understood

by investigators and their readers.

Not recommended when the reference treatment

is not well established, or is inconsistent when

compared with placebo

An appropriate sample size for noninferiority trials is

usually larger than that required for superiority trials

Noninferiority Trial DesignTo correctly interpret the results of a noninferiority

trial, we must first understand its specific

methodology and design requirements.

The hypothesis is the opposite of classically designed superiority studies.

Null hypothesis: One treatment is superior to the

other.

Alternate hypothesis: The experimental

treatment is no worse than the comparator with

respect to the prestated noninferiority margin.

Noninferiority Trial Design

Typical trial procedures such as randomization and blinding

apply; however, stricter limitations are placed on the active

control.

Comparator must be a well-established intervention with at least

one superiority trial establishing its clinical advantage over

placebo.

Overall study design is structured as closely as possible to that

superiority trial (inclusion/exclusion criteria, outcome measures,

trial conduct, etc.)

A noninferiority margin must be determined prior to trial

initiation.

Justification for the margin is provided, stating both statistical

and clinical validation.

Noninferiority Trial Design

The noninferiority margin should be no larger than the

“smallest value representing a clinically meaningful

difference” between the two interventions.

Ideally a value smaller than the minimum difference

between the active comparator and a placebo.

For example, if the referenced superiority trial showed the

comparator to be 12% better than placebo, a researcher

might set the noninferiority margin at 7%

In other words, 5% worse than the comparator, but still 7%

better than placebo

Noninferiority margin, Confidence Interval and desired

Power all influence sample size.

Noninferiority Trial Design

Intention-To-Treat (ITT) analysis can shift bias towards a

finding of noninferiority when dropouts & protocol violations

occur

The bias introduced by Per-Protocol analysis is less

predictable, especially when the rates & reasons for patient

loss differ between treatment groups.

Noninferiority trials should employ & report both methods of

analysis.

Balanced conclusions may be derived when both

methods are treated with equal importance.

A more conservative approach would be to base

conclusions on the more pessimistic of the analyses.

Noninferiority Trial Design

Noninferiority Trial Design

The tinted area represents the zone of inferiority.

•When the entire CI is less than zero, the treatment is clearly superior•When the upper limit of the CI is less than ∆, the treatment is noninferior•When the upper limit of the CI is greater than ∆, the result is inconclusive•When the entire CI is greater than ∆, the treatment is clearly inferior

Noninferiority Trial Interpretation

Superior

Noninferior

Noninferior

Inconclusive

Inconclusive

Inferior

Common ErrorsReviewers have found examples of repeated design

and reporting flaws. Becoming familiar with the

more typical faults will make it easier to evaluate

quality studies & draw appropriate conclusion.

Improper or misleading terminology

Quality reporting uses proper vocabulary, such as “Treatment

A is not inferior to (or is equivalent to) Treatment B with regard

to the margin predetermined as ____.”

Inappropriate noninferiority margin Must be defined prior to trial onset.

Justification must include be statistically & clinically relevant

Confused error types Type I error = an inferior treatment is accepted as noninferior

Type II error = a noninferior treatment is mistakenly rejected

Common errors to watch for

Incomplete analysis of results

Quality articles include both Intention-To-Treat and Per-

Protocol analyses, enabling the reader to correctly interpret

the evidence and form appropriate conclusions.

Noninferiority claims from a superiority trial

When articles omit pertinent information about the

noninferiority margin, or the sample size is calculated without

reference to that margin, clinicians should treat the

subsequent information with some skepticism

Common errors to watch for

The publication of noninferiority trials is gaining in

frequency. Evidence from these trials influence

current treatment policies, and play a role in the

decision making process for new drug acceptance.

Healthcare providers involved in the

recommendation or selection of evidenced-based

interventions should have a clear understanding of

how to evaluate the quality of these studies and

interpret their results.

 

Summary

The quality of noninferiority studies depends on their adherence

to basic design requisites and reporting obligations.

Standard trial procedures (randomization, blinding, etc)Active comparator that is well-established backed by at least one superiority trial Overall study design mirrors reference superiority trialA prestated noninferiority margin that is statistically and clinically justifiedSample size based on noninferiority margin, Confidence Interval size and desired study PowerBoth Intention-To-Treat and Per-Protocol analyses used and reportedResults reported as Confidence IntervalsConclusions drawn from comparison of Confidence Intervals with the noninferiority marginClaims of noninferiority correspond to the results provided and are written in precise, standardized vocabulary

Summary

1. Henanff AL, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1147-51

2. Dasgupta A, Lawson KA, Wilson JP. Evaluating equivalence and noninferiority trials. Am J Health-Syst Pharm. 2010;67:1337-43

3. Gotzsche PC. Lessons from and cautions about noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1172-74

4. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJW. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152-60

5. Suda KJ, Hurley AM, McKibbin T, Motl Moroney SE. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011;31(9):833-839

6. Greene WL, Concato J, Feinstein AR. Claims of equivalence in medical research: are they supported by the evidence? Ann Intern Med. 2000;132:715-722

References