Learning from issues in assay cross-validation / method transfers

Learning From Issues in Assay Cross-Validation / Method Transfers

Peter van AmsterdamAbbott Healthcare Products, Weesp, Netherlands

Presented at: The 5th Bioanalysis WorkshopApril 13-14, 2011, Montreal, Canada

Assay Cross-Validation/Method Transfers Date: 14 April 2011

2© 2011 Abbott

Contents

1. The Landscape

2. The Regulations

3. A few case studies

4. Method Transfers

5. Cross-Validation

6. Global Trials: blank matrices

7. Global Trials: OTC/Co-med interference

8. Questions / Discussion


3© 2011 Abbott

The Landscape

ToMethod Transfers

CRO PharmaCRO Y (site to site) N (rare)FromPharma Y (contracting) Y (site to site)

Cross-ValidationsComparatorCRO Pharma

Reference CROAlways, Never or Depends?

Pharma


4© 2011 Abbott

The Landscape

Contracting Site to SiteMethod Consistency

Pharma CRO CRO Pharma

Only 1 site active (move methods) Free, Similar, Identical or Depends?

Multiple sites active (expand business, capacity, regions) Free, Similar, Identical or Depends?

Role of CROContracting OutIncidental Regular

Pharma Innovative Increasing Importantanceof the Role of the CROGeneric


5© 2011 Abbott

Quiz (1)

• Who is (ultimately) responsible?

– Sponsor– CRO

• Who will be inspected?

– Sponsor– CRO

• Who pays the largest price for incompliance?

– Sponsor– CRO

FDA: Guidance for IndustryBioanalytical Method Validation

B. Partial Validation

Partial validations are modifications of already validated bioanalytical methods. Partial validation can range from as little as one intra-assay accuracy and precision determination to a nearly full validation. Typical bioanalytical method changes that fall into this category include, but are not limited to:

· Bioanalytical method transfers between laboratories or analysts


6© 2011 Abbott

EMA: Guideline On Validation Of Bioanalytical Methods

4.2 Partial validation

In situation where minor changes are made to an analytical method that has already been validated, a full validation may not be necessary, depending onto the nature of the applied changes. Changes for which a partial validation may be needed include transfer of the bioanalytical method to another laboratory, change in equipment, calibration concentration range, storage conditions etc. All modifications should be reported and the scope of revalidation or partial validation justified.

In most cases, provision of additional accuracy and precision data or relevant additional stability data on the modified issue may be sufficient.


7© 2011 Abbott

Guidance for IndustryBioanalytical Method Validation

C. Cross-Validation

Cross-validation is a comparison of validation parameters when two or more bioanalytical methods are used to generate data within the same study or across different studies. An example of cross-validation would be a situation where an original validated bioanalytical method serves as the reference and the revised bioanalytical method is the comparator. The comparisons should be done both ways.

When sample analyses within a single study are conducted at more than one site or more than one laboratory, cross-validation with spiked matrix standards and subject samples should be conducted at each site or laboratory to establish interlaboratory reliability. Cross-validation should also be considered when data generated using different analytical techniques (e.g., LC-MS-MS vs. ELISA) in different studies are included in a regulatory submission.


8© 2011 Abbott

EMA: Guideline On Validation Of Bioanalytical Methods

4.3 Cross validation

Where data are obtained from different study sites, comparison of those data is needed, and a cross validation of the applied analytical methods should be carried out. Differences in sample preparation or the use of another analytical method may result in different outcomes between the study sites. Cross validation should be performed in advance of study samples being analysedif possible. For the cross validation, the same set of QC samples should be analysed by both analytical methods. The outcome of the cross validation is critical in determining whether the obtained data are reliable and whether they can be compared and used. The difference between the two measurements should not exceed 15%.


9© 2011 Abbott

Regulatory: Conclusions

• Method TransferPartial Validation

– Accuracy & Precision– Other: depends

• (Essentially) different methods during drug development or muliplelabs in one study

Cross-Validation

– QCs & Samples (FDA)– QCs only (EMA)


10© 2011 Abbott

Quiz (2)

• Is Cross-Validation required for Pop-PK, PK/PD, S&M (pool data frommultiple studies)?

– Yes– No– Depends


11© 2011 Abbott


12© 2011 Abbott

Case: GC-ECD 3-keto-delta-4,6 Steroid (1990)

• Requirements:

– LLOQs: Parent 0.1 ng/ml, active metabolite 0.5 ng/ml– Range: At least 2 decades– No interference from endogenous steroids

• In house GC-MS instrument capacity to limited GC-ECD method

• Large volume and high time pressure clinical program launched

• Contracting out to 3 ~mid sized CROs

• Complicated and laborious assay.

– L/L and HPLC clean-up with fraction collection.– Required freshly distilled solvents– 2 analytes, 2 IS, 2 different GC set-ups


13© 2011 Abbott

Case: GC-ECD 3-keto-delta-4,6 Steroid (1990)

• Multiple studies running at multiple labs in parallel– Sponsor wanted to be in full control – Daily reporting of CAL & QC data + faxing selected chromatograms

• Issue: method only ran well on sponsor’s HP 5890.

– Less well on CRO’s HPs (2 CROs)• Troubleshooting and training of CRO staff by sponsor• Set-up solvent distillations

– Hardly at all on Varians (1 CRO)• Ship 2 sponsor’s HP 5890 to CRO• Training of CRO staff by sponsor• Set-up solvent distillations

Nightmare Method Transfer


14© 2011 Abbott

Case HPLC-EC Aryl-piperazine (1995)

• Requirements:

– LLOQ: Parent 0.1 ng/ml– Highly selective (multiple metabolites)– Range: 3 decades

• Early 90s

– HPLC-FLU failed, GC-NFID failed, GC-ECD failed. – Extremely complicated LC-GC method involving a combined hydrolysation /

derivatisation step worked, however only 1 column manufacturer and judged not-transferable: discontinued.

• New method HPLC-EC, but with extensive L/L and SPE sample clean-up and column swithing

• Large volume pre-clin & clinical program. Contracting out to 2 CROs

• Demand: exact copy of sponsor method


15© 2011 Abbott

Case HPLC-EC Aryl-piperazine (1995)

• CRO1: FAIL, CRO2: FAIL (and fail again)– Troubleshooting at CRO staff by sponsor

– Restart: CRO1: FAIL, CRO2: FAIL

– Extensive in-house training CRO technicians

– ReRestart: CRO1: OKish (large % batch failure), CRO2: OKish (background / positive blanks)

• CRO3: First time right and hardly ever failed a batch

• CRO4: OKish. – Strong desire for variation on method. Chrom & QC data looked very good.

– BUT: Indications of issues with separation from metabolites.

Nightmare Method Transfer

Case LC-MS/MS Benzoxazol-piperazine (2000)

• Requirements:

– LLOQ: Parent 0.02 ng/ml– Range: At least 3 decades

• Not very complicated or laborious assay, but challenging LLOQ

• After FIH, clinical studies and long term Tox to be contracted out.

– Demand: similar to sponsor method

• Problem free study conduct:

– in-house, CRO1 (until change of personnel), CRO2 (entire program), US-site (NA studies only), CRO3, co-development partner

• Except…

– BA lab of Tox CRO never, really never, got the method performing well for more then 2 batches.


16© 2011 Abbott

Dream Method Transfer

Case CTD of old drug (2005)

• Lypophilic aliphatic t-amino compound. No UV absorbance. No derivatization possible.

• Long development history

• Multiple methods and labs

– GC-FID, GC-NFID– Direct MS, GC-MS with some random IS, GC-MS with analogue IS– LC-MS with analogue, LC-MS/MS with deuterated IS

• Local validations only

– Oldest work: very limited

• No cross-validations done


17© 2011 Abbott

Challenging 2.7.1


18© 2011 Abbott

Method Transfers: Lessons Learned

• ‘Green Fingers’ methods are hard to transfer

– Match equipment, chemicals, consumables, glassware as much a reasonably possible

– Thorough training of CRO staff by Sponsor at CRO and/or at Sponsor site.– Exact copy of a method is no guarantee for success

• ‘Standard’ methods are relatively easy to transfer

– Match types of equipment, chemicals, consumables & glassware– Allow local adaptations to the method as long as they stay within the basic

principles of the reference method.– Be ready for failure and have a back-up scenario.

• Strive for consistency in methods over time

– Do Cross-Validations when methods change in their principles

Method Transfers: Validation

• Within lab (method upgrades, same sample work-up or measurement principles)

– Partial validation (accuracy & precision batches)

• From lab to lab (similar and identical methods)

– Full validation except long term stability & specificity/selectivity

• From method to method (different sample work-up or measurement principles)

– Full validation including long term stability & specificity/selectivity


19© 2011 Abbott

Cross-Validations (Interlab)

• Originator lab / method is the reference lab / method

– Regardless whether it is CRO or Sponsor

• Reference lab prepares test sample sets

– 20 Spiked Blanks (QCs)• Spanning the entire range. May be Routine QCs & Freshly prepared

– 20 Pooled Study Samples• Spanning the entire range

– Samples are blinded– Spare sets for possible future cases

• Labs analyze 2 sets. Usual run acceptance criteria. Create analysis report

• Reference lab writes interlab report

• Criteria:

– CV duplicates < 15%– Slope between 0.85 and 1.15


20© 2011 Abbott

Quiz (3)

• A Cross-Validation shows substantial differences between two techniques or labs, but perfectly correlated. Would you accept?

– Yes– No– Depends


21© 2011 Abbott

Global Studies: Blank Matrices

• Should we match the blank matrix with the regional origin of thesample?

• Considering:– Normal ranges on many many blood parameters span on average about

one decade– Blank matrices are different from person to person– There are no practical differences between populations in common lab

values (clinical chemistry)– Differences between males and females are larger then between any

other groups– Cultural differences, especially eating habits and types of food are of

larger impact on the matrix then genetic differences.

• My Answer: No


22© 2011 Abbott

Reference Ranges

Source: http://en.wikipedia.org/wiki/Reference_ranges_for_blood_tests


24© 2011 Abbott

Global Studies: OTC interference evaluation

• Testing on OTC interference is commonly done by a number of labs– Acetaminophen, Ibuprofen, Levonorgestrel, Nicotine, Caffeine, ....– Standard list varies from lab to lab

• Should we keep doing this? Extent? Or stop?

• Considering:– Country to country differences in OTC drugs – Difficult and quite costly to include OTC drug metabolites– Herbal products containing of 100s of compounds in various compositions– Influence of food on plasma and urine composition– Very selective tools: L/L or SPE + LC-MS/MS

• My Answer: No

Targeted OTC and Co-medication interference testing

During drug development:

• Day-by-day: check on used co-medication or OTC when for a given subject:

– Analyte levels are substantially different from the general picture– IS response substantially differs from other samples– Broad, odd or extra peaks are detected

• Early phase III: obtain listings on used OTC drugs and any othermedication used during clinical trials.

– Evaluate list of compounds with respect to the likeliness to interfere• Molecular weight, Structure, Concentrations, Chemicophysical properties

– Test selected potentially suspect compounds at relevant concentrations in the assay


26© 2011 Abbott

Questions !

• CRO <-> Sponsor– Who is responsible: sponsor or CRO?– Who will be inspected: sponsor or CRO?– Who pays the largest price for incompliance: sponsor or CRO? – Sponsor SOPs or CRO SOPs: yes, no or depends?

• Cross-Validation– Is Cross-Validation required for Pop-PK studies: yes, no, depends?– What if when the comparator is not matching the reference lab data?– Transferred assays must be 1-to-1 copies of the original: yes, no, depends– Transferred methods must cross-validated: always, never, depends

• Multi-site / Global– Should we match the blank matrix with the regional origin of the sample?– Should we take the international OTC market into account when doing OTC

interference tests?

Learning from issues in assay cross-validation / method transfers

Science

Transcript of Learning from issues in assay cross-validation / method transfers