Strategies for Analytical Method Replacements for Approved ...c.ymcdn.com/sites/ · Strategies for...
Transcript of Strategies for Analytical Method Replacements for Approved ...c.ymcdn.com/sites/ · Strategies for...
Stephan O. Krause, PhDWCBP 2017, Washington, DC
Strictly Confidential
24-26 January 2017
Strategies for Analytical Method Replacements
for Approved Products
Outline
I. Introduction and Method Comparison Strategies
Replacement strategy background and rationale
General Strategies for Qualitative and Quantitative Methods
II. Analytical Method Comparison (AMC)
Case Studies (qualitative and quantitative methods)
III. Comparison and Implementation considerations
The content and views expressed by the author/presenter are not necessarily the views of the organization he represents.
3
Analytical Method Comparability (AMC)
What is AMC ?
– AMC is the demonstration of comparable (“equivalent or better”) test method
performance of a modified/new method.
– AMC should be demonstrated for methods replacing approved methods (in-
house licensed, compendial, or otherwise recognized).
Why is it important ?
– A continuous suitable test method performance must be assured for
safety/efficacy/quality to patients (linking to licensed specs and clinical data).
– This also assures a data/quality continuum for the sponsor.
How exactly can we demonstrate AMC ?
– Follow ICH E9 and CPMPs Points to Consider guidelines (detailed in PDA TR
57)
– Demonstrate “equal or better” by testing for non-inferiority, equivalence, or
superiority depending on assay type and need (risk).
– Compare particular test method performance criteria per ICH Q2(R1) (detailed
in PDA TR 57).
Krause/PDA, 2012
4
Points to Consider for AMC Studies
Krause/PDA, 2012
FTIH POC BLAQTPP
Approval
CQ
A
De
ve
lop
me
nt
Sp
ec
s L
ife
Cy
cle
Mg
mt
CM
C a
nd
Te
ch
Tra
ns
fer
Pro
ce
ss
Analytical
Manufacturing
Strategic or
Tactical ChangesDose
change
Delivery
Device
PPQ lotsMfg
TransferMfg Process
Change
Qualification
Process Verification (CPV)
Global
Supply
Development of Control Strategy
Specification Life Cycle Management
ValidationOptimization Method ChangeContinuous VerificationS
up
po
rt PP
Q S
pe
cs
?Transfer
5
Points to Consider for AMC Studies
Krause/PDA, 2012
Consider:
A. Impact on DS/DP Specifications
FTIH POC BLAQTPP
CQ
A
De
ve
lop
me
nt
Sp
ec
s L
ife
Cy
cle
Mg
mt
CM
C a
nd
Te
ch
Tra
ns
fer
Pro
ce
ss
Analytical
Manufacturing
Strategic or
Tactical ChangesDose
change
Delivery
Device
PPQ lotsMfg
TransferMfg Process
Change
Qualification
Process Verification (CPV)
Global
Supply
Specification Life Cycle Management
ValidationOptimization Method ChangeContinuous VerificationS
up
po
rt PP
Q S
pe
cs
?Transfer
6
Points to Consider for AMC Studies
Krause/PDA, 2012
Consider:
A. Impact on DS/DP Specifications
B. Impact on CPV program (CPPs, KPPs)
FTIH POC BLAQTPP
CQ
A
De
ve
lop
me
nt
Sp
ec
s L
ife
Cy
cle
Mg
mt
CM
C a
nd
Te
ch
Tra
ns
fer
Pro
ce
ss
Analytical
Manufacturing
Strategic or
Tactical ChangesDose
change
Delivery
Device
PPQ lotsMfg
TransferMfg Process
Change
Qualification
Process Verification (CPV)
Global
Supply
Specification Life Cycle Management
ValidationOptimization Method ChangeContinuous VerificationS
up
po
rt PP
Q S
pe
cs
?Transfer
7
Points to Consider for AMC Studies
Krause/PDA, 2012
Consider:
A. Impact on DS/DP Specifications
B. Impact on CPV program (CPPs, KPPs)
C. Impact of ROW Lot Release Testing
FTIH POC BLAQTPP
CQ
A
De
ve
lop
me
nt
Sp
ec
s L
ife
Cy
cle
Mg
mt
CM
C a
nd
Te
ch
Tra
ns
fer
Pro
ce
ss
Analytical
Manufacturing
Strategic or
Tactical ChangesDose
change
Delivery
Device
PPQ lotsMfg
TransferMfg Process
Change
Qualification
Process Verification (CPV)
Global
Supply
Specification Life Cycle Management
ValidationOptimization Method ChangeContinuous VerificationS
up
po
rt PP
Q S
pe
cs
?Transfer
8
Analytical Method Comparability StudiesSuggested Performance Characteristics and Statistics
ICH Q2(R1) Category
Identification and/or
Pass/Fail Test (Qualitative)
Limit Test (Qualitative)
Limit Test (Quantitative)
Potency or Content (Purity
or Range) (Quantitative)
Accuracy Not Required Not Required Paired matched T-Test (TOST); Some Data could be at QL and/or at OOS level
TOST
Intermediate Precision
Not Required Not Required Mixed linear model; F-test statistics
Mixed linear model; F-test statistics
Specificity Proportions Test; Chi-Squared (Fisher Exact Test) for Number of Correct Observations
Proportions Test; Chi-Squared (Fisher Exact Test) for Number of Correct Observations
Not Required Not Required
Detection Limit
Not Required Depends on how DL was established. Proportions Test calculations may be used
Not Required Not Required
QuantitationLimit
Not Required Not Required Depends on how QL was established.
Not Required
Krause/PDA, 2012
9
AMC Categories (from ICH E9/PDA TR 57)
Qualitative Methods
• Non-inferiority
• Superiority
Quantitative Methods
• Equivalence
Krause/PDA, 2012
10
Demonstrating Non-Inferiority
No
n-I
nfe
rio
rity
Lim
it
-9
0%
CI
- Delta 0
+9
0%
CI
Current Method
“Better Results”
New Method
“Better Results”No difference
Me
an
Diffe
ren
ce
Non-inferiority Testing (PDA TR 57)Current Method = Reference
Non-Inferiority Demonstrated
Desirable Direction/Range
Krause/PDA, 2012
11
Demonstrating Superiority
-9
0%
CI
0
+9
0%
CI
Current Method
“Better Results”
New Method
“Better Results”No differenceM
ea
n D
iffe
ren
ce
Superiority Testing (PDA TR 57)Current Method = Reference
Superiority Demonstrated
Su
pe
rio
rity
Lim
it (
0)
Desirable Direction/Range
Krause/PDA, 2012
12
Qualitative MethodsDemonstrating Non-Inferiority
A faster and technologically advanced method for (upstream in-process) sterility testing was validated and compared to the compendial EP/USP Sterility Test.
The current proportions (hit/miss ratio) of the USP/EP method is approx. 77% (23% false negative results with SD of approx. 5.0%).
Using a Proportions Test, the non-inferiority comparison at the 95% confidence level (p=0.05) was chosen with a pre-specified non-inferiority limit of –10.0%.
Justification for Non-Inferiority Model and Limit:
Non-inferiority, equivalence, and superiority are all acceptable outcomes.
The increased testing frequency of daily (n=7 per week) and faster results for the new sterility versus twice weekly (n=2 per week) for the EP/USP Sterility test significantly increases the likelihood of detecting organisms with the new method.
The -10.0% limit versus the compendial (current) method was set and justified based on the compendial method performance (2 x EP/USP method standard deviations: 2x 5.0% = 10.0%) for the detection of (pooled) reference microbial organisms and/or plant isolates.
Krause/PDA, 2012
13
Demonstrating Non-InferiorityResults for the Non-Inferiority Test: Candidate Method vs. USP Sterility
Krause/PDA, 2012
Method Positives Total Samples (n) Positives-to-Fail Ratios
Candidate 225 300 0.75 (75%)
EP/USP 232 300 0.77 (77%)
Statistical Results
Difference = p (new method) - p (EP/USP)
Estimate for difference: -0.023 (-2.3%)
95% lower confidence interval limit for difference: -0.080 (-8.0%) (Limit = -10.0%)
14
Demonstrating Non-Inferiority
No
n-I
nfe
rio
rity
Lim
it
-9
0%
CI
- 10.0% 0
+9
0%
CI
Current Method
“Better Results”
New Method
“Better Results”No difference
Me
an
Diffe
ren
ce
Non-inferiority Testing (PDA TR 57)Current Method = Reference
Non-Inferiority Demonstrated
Desirable Direction/Range
Krause/PDA, 2012
15
Demonstrating Superiority
From the previous example for non-inferiority: When the relative testing frequency of our example of n=7 (new method) versus n=2 for the compendial method is integrated in our comparison studies, the superiority of the new method could be demonstrated.
Krause/PDA, 2012
16
Demonstrating Superiority
Results
Candidate Method (7x) vs. EP/USP Sterility (2x): Sample Positives Total Probability 95% CI for Probability Candidate 225 300 0.9999 0.9997 – 1.0000 EP/USP 232 300 0.947 0.921 – 0.967
Krause/PDA, 2012
Results/Conclusions:
Superiority at the 95% confidence level could be demonstrated because the new
method’s 95% confidence interval (0.9997-1.0000) for the positive-to-fail
probability (0.9999) lies entirely to the right of the 95% confidence interval
(0.921-0.967) of the compendial method’s positive-to-fail probability (0.947).
The superiority test was passed with a much greater relative margin than the
non-inferiority test. This is a good example why we should always consider
upfront which comparison study to select and how to defend our strategy in
regulatory submission.
17
Superiority of New Method (vs. Current/Compendial)
Demonstrated
Krause/PDA, 2012
92.1
%
94.7
%
91% 92% 93% 94% 95% 96% 97% 98% 99% 100%
96.7
%
USP/EP Method
“Better”
Candidate Method
“Better”
USP/EP
Probability:
99.9
7%
99.9
9%
10
0.0
0%
Candidate Method
(not drawn to scale)
18
Demonstrating Equivalence
Eq
uiv
ale
nce
Lim
it
-9
0%
CI
- 0.50% 0 + 0.50%
+9
0%
CI
New Method
“Lower Results”
New Method
“Higher Results”
Eq
uiv
ale
nce
Lim
it
No difference
Me
an
Diffe
ren
ce
Equivalence Testing (PDA TR 57)Current Method = Reference
Equivalence Demonstrated
Krause/PDA, 2012
19
Demonstrating EquivalenceIntroduction
It was decided to develop and validate a capillary electrophoresis (CE) method to replace a current SDS-PAGE electrophoretic method The method performance characteristics (quantitative limit test), are compared:
- Accuracy (“matching”) is directly compared through product sample testing (release and stability).
- (Intermediate) precision is directly compared through (recent) assay control comparison.
- Quantitation limits are compared historically from validation studies
For accuracy/matching: A delta of plus/minus 0.50% was chosen for the equivalence category between both impurity levels from the analysis of historical data with respect to the current specifications (for SDS-PAGE). Both methods were run simultaneously (side-by-side) for each of a total of n=30 reported results were compared by two-sided matched-paired t-test statistics with pre-specified equivalence limits of plus/minus 0.50% (% = reported percent and not relative percent).
Krause/PDA, 2012
20
Demonstrating EquivalenceIntroduction
It was decided to develop and validate a capillary electrophoresis (CE) method to replace a current SDS-PAGE electrophoretic method The method performance characteristics (quantitative limit test), are compared:
- Accuracy (“matching”) is directly compared through product sample testing (release and stability).
- (Intermediate) precision is directly compared through (recent) assay control comparison.
- Quantitation limits are compared historically from validation studies
For accuracy/matching: A limit of plus/minus 0.50% was chosen for the equivalence category between both impurity levels from the analysis of historical data with respect to the current specifications (for SDS-PAGE). Both methods were run simultaneously (side-by-side) for each of a total of n=30 reported results were compared by two-sided matched-paired t-test statistics with pre-specified equivalence limits of plus/minus 0.50% (% = reported percent and not relative percent).
Krause/PDA, 2012
21
Evaluating Process CapabilityDP Release and Stability
Time (years)
Current Method(Purity %)
1 2 3 4
100.0%
98.5%
DP EOSL NLT 97.0%
99.5%
99.0%
98.0%
97.0%
Pooled n=6 DP lots (R -EOSL = 0.60%)
Predicted R – EOSL Difference:Long-term assay variation + pooled slope
uncertainty (n=6 DP)
Mean +/- 3 SDs
Historical DP Release
Results (n=40)
DP Release
22
100.0%
98.5%
DP EOSL
99.5%
99.0%
98.0%
97.0%
Historical DP Release
Results (n=40)
DP Release
Mean +/- 3 SDs
0.5%
0.4%
CpK = 1.00 (- 3SDs)
4-Year Loss
(0.6%)
Predicted DP Stability (n=6)
Evaluating Process CapabilityDP Release and Stability
23
Setting Risk-Based AMC Acceptance CriteriaOptions/Examples
100.0%
98.5%
DP EOSL
99.5%
99.0%
98.0%
97.0%
DP Release
Mean +/- 3 SDs
0.50%
0.40%
CpK = 1.00 (- 3SDs)
4-Year Loss
(0.60%)
Options/Examples for Equivalence Limit(s) for 90% CI for Mean Difference(s): 1. Symmetrical limit(s) based on current mfg/testing capability: DP R – CpK 1.00 = +/- 0.50% - Pool release and stability data (if possible; may need to normalize data)- Evaluate separately (ex., for release: +/- 0.50%; for EOSL: +/- 0.40%)- Accept stability comparison results if < 0.40% different (90% lower CI)
2. Asymmetrical limit(s): -0.50% (protect mfger) and < +0.50% (protect patient)- ...etc.
24
Setting Risk-Based AMC Acceptance CriteriaSymmetrical Margins (for Release)
100.0%
98.5%
DP EOSL
99.5%
99.0%
98.0%
97.0%
DP Release
Mean +/- 3 SDs
0.50%
0.40%
CpK = 1.00 (- 3SDs)
4-Year Loss
(0.60%)
Options/Examples for Equivalence Limit(s) for 90% CI for Mean Difference(s): 1. Symmetrical limit(s) based on current mfg/testing capability: DP R – CpK 1.00 = +/- 0.50% - Pool release and stability data (if possible; may need to normalize data)- Evaluate separately (ex., for release: +/- 0.50%; for EOSL: +/- 0.40%)- Accept stability comparison results if < 0.40% different (90% lower CI)
2. Asymmetrical limit(s): -0.50% (protect mfger) and < +0.50% (protect patient)- ...etc.
25
Setting Risk-Based AMC Acceptance CriteriaSymmetrical Margins (for Release)
100.0%
98.5%
DP EOSL
99.5%
99.0%
98.0%
97.0%
DP Release
Mean +/- 3 SDs
0.50%
0.40%
CpK = 1.00 (- 3SDs)
4-Year Loss
(0.60%)
Options/Examples for Equivalence Limit(s) for 90% CI for Mean Difference(s): 1. Symmetrical limit(s) based on current mfg/testing capability: DP R – CpK 1.00 = +/- 0.50% - Pool release and stability data (if possible; may need to normalize data)- Evaluate separately (ex., for release: +/- 0.50%; for EOSL: +/- 0.40%)- Accept stability comparison results if < 0.40% different (90% lower CI)
2. Asymmetrical limit(s): -0.50% (protect mfger) and < +0.50% (protect patient)- ...etc.
Demonstrating Equivalence
Eq
uiv
ale
nce
Lim
it
-9
0%
CI
- 0.50% 0 + 0.50%
+9
0%
CI
New Method
“Lower Results”
New Method
“Higher Results”
Eq
uiv
ale
nce
Lim
it
No difference
Me
an
Diffe
ren
ce
Equivalence Testing (PDA TR 57)Current Method = Reference
Equivalence Demonstrated
Krause/PDA, 2012
27
AMC Study ResultsSymmetrical Limits (for Release)
Time (years)1 2 3 4
100.0%
98.5%
DP EOSL NLT 97.0%
99.5%
99.0%
98.0%
97.0%
Avg of n=6 DP lots (R - EOSL = 0.60%)
N=30
DP Release
N=30
Avg of n=3 DP lots (R - EOSL = 0.90%)
Mean Difference = -0.25% (R)
Mean Difference = -0.55% (R + EOSL)
Current
Method
(+/- 3 SDs)
New
Method
(+/- 3 SDs)
28
AMC Study ResultsSymmetrical Limits (for Release)
Time (years)1 2 3 4
100.0%
98.5%
DP EOSL NLT 97.0%
99.5%
99.0%
98.0%
97.0%
Avg of n=6 DP lots (R - EOSL = 0.60%)
N=30
DP Release
N=30
Avg of n=3 DP lots (R - EOSL = 0.90%)
Mean Difference = -0.25% (R)
Mean Difference = -0.55% (R + EOSL)
CpK > CpK
29
Equivalence of New Method Demonstrated
Krause/PDA, 2012
Eq
uiv
ale
nce
Lim
it
- Delta 0 + Delta
New Method
“Lower Results”
New Method
“Higher Results”
Eq
uiv
ale
nce
Lim
it
No difference
- 9
0%
CI
+ 9
0%
CI
- 9
0%
CI
+ 9
0%
CI
- 9
0%
CI
+ 9
0%
CI
- 9
0%
CI
+ 9
0%
CI
- 9
0%
CI
+ 9
0%
CI Passes Equivalence
(stat. different)
Passes Equivalence
(stat. not different)
Passes Equivalence
(stat. different)
Equivalence Unclear
(stat. different)
Fails Equivalence
(stat. different)
30
Points to Consider for AMC Studies and New
Method Implementation
• The comparison category should be justified.
– For example, a non-inferiority test may be suitable, if all outcomes (non-inferiority,
equivalence, and superiority) are acceptable, and if the new method is superior in other
aspects such as faster test results and/or increased sampling/testing.
• The pre-specified maximum allowable difference(s) should be justified.
The difference limit(s) should strike a balance among (potentially)
opposing incentives:
– Impact on patient and/or manufacturing.
– Passing AMC results as “comparable” when they are not, and vice versa.
Krause/PDA, 2012