Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying...

54
Commissie voor Klinische Biologie Commission de Biologie Clinique Quality Assurance in PCR testing Greet Ieven 25.03.2009 Rue Juliette Wytsmanstraat 14 | 1050 Brussels | Belgium T +32 2 642 51 11 | F +32 2 642 50 01 | email: [email protected] | www.iph.fgov.be

Transcript of Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying...

Page 1: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Commissie voor Klinische BiologieCommission de Biologie Clinique

Quality Assurance in PCR testing

Greet Ieven25.03.2009

Rue Juliette Wytsmanstraat 14 | 1050 Brussels | BelgiumT +32 2 642 51 11 | F +32 2 642 50 01 | email: [email protected] | www.iph.fgov.be

Page 2: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

The evolution of Molecular technology: ‘water bath to chip…..’

1985

PCR Amplification

1st PCR kits (HLA)

Semi-automated systems

Real Time

1990 1995 2000

1st DNA PCR kits for infectious disease

Automated Extraction systems

NAT-CHIPS

Page 3: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

In vitro NAAT’s

• Polymerase Chain Reaction (PCR)

• Branched DNA (bDNA)

• Transcription Mediated Amplification (TMA)

• Nucleic Acid Sequence Based Amplification (NASBA)

• Strand Displacement Amplification (SDA)

• Hybrid Capture (HC)

To name a few……..

Page 4: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Practical application in detecting & identifying infectious agents(unculturable organisms).

Identification & characterisation of antimicrobial resistance

Support the clinical treatment process (which treatment, extent, and duration)

Monitoring the course of infection (Therapeutic monitoring)

Epidemiologic investigations (socially important diseases, healthrisks).

Discovery of new viruses(HCV, HHV8, Hendra, Nipah, hMPV, SARS-CoV, HCoV-NL63, HCoV-HKU, BOCA virus,…)

The molecular revolution in Infectious Diseases

Diagnosis Prevention Prognosis Therapy

Page 5: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Commercially Versus Not-Commercially Available Assays in Clinical Virology

Mainly In-house assaysHIV-2EBVCMVHSV VZVHHV type 6HHV type 8EnterovirusRhinovirusRSVHuman metapneumovirusInfluenzavirusParainfluenzavirusWest Nile virusParvovirus B19Hepatitis A virusNorovirusLassa virusVariolaSARS………………………………

Commercially Available

HIV-1

HCV

HBV

CT / Ngo

CMVEBVHSVetc

Page 6: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

37.118.07.651.775.070.911.27.021.5Enterovirus (EV)

37.3--48.0--14.7--Varicella-Zoster Virus (VZV)

55.843.83.627.937.273.716.319.027.7Human Cytomegalovirus (CMV)

50.533.922.138.356.973.211.29.24.7Herpes simplex virus (HSV)

47.741.8-29.628.3-22.729.9-Epstein-Barr virus (EBV)

0.0--100.0--0.0--HIV (DNA) Pilot

5.94.40.08.05.67.586.190.092.5Human Immunodeficiency virus

n/a--26.1--73.9--HCV Genotyping

13.24.81.612.27.217.174.688.081.3Hepatitis C virus (HCV)

30.024.00.029.025.039.041.052.061.0Hepatitis B virus (HBV)

200320022001200320022001200320022001

Real-timeConventional

In-house assaysCommercial assaysProgramme Title

Increasing use of real-time PCR…

Page 7: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Types of NAAT assay used for the detectionof HSV in UK between 1999 and 2003

Key Points

• Increase in the use of Real Time PCR

• Decrease in the use of gel based methods

• Nested PCR remains most common

• Increase in the reporting of semi-quantitative / quantitative data

• Geographical spread of methods0%

50%

100%

1999 2000 2001 2002 2003

YearIn-house Single PCR In-house Nested PCRIn-house Real Time PCR In House other / Not Specified

Source: QCCA / QCMD HSV distributions 1999 - 2003

Page 8: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Increasing use of ‘quantification’

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

2004 2005 2006 2007 2008

%

Year

Commercial assays

Real time assays

Participants reporting real time PCR for CMV detection

Page 9: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

The key issues

• Molecular, particularly real-time PCR, replacing traditional diagnostic methods in clinical diagnostics laboratories

• Microbial load a primary diagnostic tool for many current & developing targets.

• Many clinically relevant targets.

• Many in-house molecular assays and QC reagents.

• In some cases the technology still has to prove itself in the clinical setting

• Lack of standardised control reagents & primary standards – large variability of results within and between laboratories

• QA is important in this rapidly moving field to ensure reliable patient test results

Page 10: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Common challenges facing molecular diagnostics

• Assay Sensitivity – Variability, relatively high detection limits of some methods

• Clinical significance - quantitative analysis / reporting / ‘setting cut-off levels’

• Variable NA isolation procedures – limited integration

• Standardisation of reagents – in-house versus commercial methods

• Standardisation & control of molecular assays – use of appropriate controls

• Lack of appropriate control material – CRMs, Calibration standards.

• General Result Reporting – units used / nomenclature used

• Regulatory environment - increasingly demanding, expanding in scope

Page 11: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

New method

Establish Performance

PerformClinical tests

Check InternalQC

Verify method

Validate Performance

Method Equivalence

Quality Improvements

Report Results

Implement method

Assay Verification

AssayValidation

Assay Implementation

EQA

Method Verification, Validation, Implementation, & Control within the Clinical Diagnostic Laboratory

Quality Management

System

Clinical Accreditation

Page 12: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Method Verification, Validation and Implementation

• Laboratory development phase: performance characteristics- accuracy: ability to produce correct result- precision: ability to consistently produce same result- analytical sensitivity: lowest amount of target NA that still

returns a positive result- analytical specificity: ability to return a negative result when

target NA is not present• Clinical validation phase:

- relating a positive test result to the presence of disease• Test implementation:

- Defining policies and procedures- Use of relevant controls, ....

Page 13: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Analytical Sensitivity and specificity

• Analytical sensitivity tested on:

• Progressively lower titrations of purified NA from target organisms

• or dilutions of pure cultures of target organisms

• preferably in background of matrix on which NAAT will be applied

• Analytical specificity tested on:

• taxonomically related organisms

- e.g. N. meningitidis and N. lactamica in case of N. gonorrhoeae PCR

• or epidemiologically related organisms

- e.g. other respiratory agents in case of Mycoplasma, Chlamydia, or

M. tuberculosis PCR

Page 14: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Validation of Clinical Sensitivityand Specificity

• Value of a new test is measured by comparison with a reference test or “gold standard”

• Sensitivity of NAAT < reference test→ validation by comparison with reference teste.g. Sensitivity of MTB PCR in sputum → PCR less sensitive:

small fraction tested in NAAT, large part in culture mediumProblems when the new test has a higher sens than ref test

• Solutions offered by:

- discrepant analysis

- expanded gold standard

- latent class analysis

Page 15: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Validation of Amplification Tests

Number of specimens to be tested depends on intended use of datae.g. 100 incoming routine specimens tested and prevalence 5%→ good sample size for evaluation of rate of false positives in

negative population→ not sufficient number of positives for evaluation of false negative resultsOPTIONS: - test specimens until desired Nr of positives

- include stored positive specimensGUIDELINE: 50 positive - 50 negative specimens

NCCLS EP 12A consensus document

Page 16: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Clinical Validaton of Amplification Methods

Sometimes difficults to obtain clinical specimens: • low prevalence

- CSF positive for HSV- synovial fluid positive for Borrelia burgdorferi

• difficult to obtain clinical material for PCR- Trophyrema whippeli in endocarditis- Toxoplasma gondii in amniotic fluid

⇒ alternative evaluation- reference panels and proficiency samples- specimens exchange if no ECP available

Page 17: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Molecular Diagnostic Tests : Proficiency Testing

• To confirm skill of lab in test performance• To ensure reproducibility• To validate amplification methods• Frequency :

- 2-3 testing events / year- 5 test samples / testing event covering full range : non reactive → highly reactive

• Samples : - whole organisms or isolated nucleic acids- previously characterized specimens- or duplicate, blinded specimens (internal consistency)

NCCLS MM3-A. , 1995CLSI MM3-2A Approved guidelines 2003

Page 18: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

New method

Establish Performance

PerformClinical tests

Check InternalQC

Verify method

Validate Performance

Method Equivalence

Quality Improvements

Report Results

Implement method

Assay Verification

AssayValidation

Assay Implementation

EQA

Method Verification, Validation, Implementation, & Control within the Clinical Diagnostic Laboratory

Quality Management

System

Clinical Accreditation

Page 19: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

The issues

• False positives- Assay contamination- Assay specificity

• False negatives- Assay sensitivity- Strain variation- Inhibition

• Quantitative accuracy- Assay/operator variation- Lack of standards

judicious choice of application to specific organismsQA measures and controls to be designed and executed

Page 20: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False positive results – late 20th Cent.

mid 1990s

Late 1990s

Page 21: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False positive results – early 21st Cent.

0

5

10

15

20

25

30

35

40

45

50

2003 2004 2005 2006 2007

%

Year

Chlamydia trachomatis A

Chlamydia trachomatis B

Neisseria gonorrhoeae

Mycobacterium tuberculosis

Legionella pneumophila

Toxoplasma gondii

MRSA

Enterovirus

Herpes simplex virus

Varicella‐Zoster virus

Hepatitis B virus

Hepatitis C virus

Parvovirus B19

HIV (RNA)

HIV (DNA) 

Epstein‐Barr virus

Human Cytomegalovirus

CMV Dried Blood Spot

Adenovirus

Influenza A / B

RSV / hMPV

Rhinovirus / Coronavirus

Parainfluenzavirus

Page 22: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

• GLP procedures (uni-directional workflow, defined working areas)•. Each with dedicated equipment, pipettors, laboratory coats

- Sample and reagent preparation, class II safety cabinet- Template addition- Amplification / detection with separate ventilation systems

pre- and post PCR areas

• Practices within lab for contamination control.- Use aerosol-guarded filtertips- Use aliquots of all reagents to limit handling- Use UNG or chemical methods to control amplicon carry-over- Use low copy-number for + controls (10-50 copies/PCR)- Use large numbers of negative controls- Change gloves between steps or when entering separate ereas- ……..

WORKFLOW

Internal QA measures within the molecular diagnostic lab

CLSI MM3-2A Approved guidelines 2003

Page 23: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Internal QA measures within the molecular diagnostic lab

• Instruments and equipment- Incubators, heating blocks, extraction instruments, thermal

cyclers, spectrophotometers,...• QA of test component reagents

- Primers and probes, buffers, dNTP’s, enzymes• Test controls

- specimen preparation, amplification, detection• positive controls: very small amounts of DNA• negative controls: at least 1 negative control every

5th - 10th DNA extraction• internal controls: should be included in each sample

CLSI MM3-2A Approved guidelines 2003

Page 24: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False positives and assay specificity

• A proportion of conventional commercial Real time PCR assays report some closely-related Neisseria spp. as N. gonorrhoeae

• Issues with closely related species (N. meningitidis and N. lactamica) and assays that target the highly conserved 16S rRNA

• Mandatory confirmatory testing should be required on all positive test results

Page 25: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False negative results

• False negatives caused by- Sensitivity issues- Strain variation- Assay inhibition

• Assays vary considerably in reported limits of detection• What is the clinically relevant level of a pathogen?

Page 26: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

C. pneumoniae Proficiency Panel:Results

Number of samples with correct results/number of samples testedLab nr Nr ofNAAT

runs/sample

Nr ofdata sets All samples Negative

samplesWeaklypositivesamples

Positivesamples

Stronglypositivesamples

1 1 1 9/10 4/4 2/3 1/1 2/21 MX 1 1 7/10 4/4 0/3 1/1 2/2

2 2 1 7/10 4/4 0/3 1/1 2/23 2 1 3/10 0/4 2/3 0/1 1/24 2 1 8/10 4/4 1/3 1/1 2/25 1 1 8/10 4/4 1/3 1/1 2/26 2 1 9/10 4/4 2/3 1/1 2/27 2 1 9/10 4/4 2/3 1/1 2/28 1 1 10/10 4/4 3/3 1/1 2/29 2 1 7/10 4/4 0/3 1/1 2/210 4 1 7/10 4/4 0/3 1/1 2/211 2 2 6/10 4/4 0/3 0/1 2/212 2 1 7/10 4/4 0/3 1/1 2/213 1 1 7/10 4/4 1/3 1/1 1/2

13 MX 1 1 6/10 4/4 0/3 1/1 1/214 MX 1 1 4/10 4/4 0/3 0/1 0/2

15 1 1 9/10 4/4 1/3 1/1 2/216 2 2 9/10 4/4 2/3 1/1 2/2

% correctresults

72.2 94.4 31.5 83.3 88.9

Page 27: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

What is the clinically relevant level of detection?

Clinical picture and herpesvirusDNA load in CSF

Virus Clinical picture Virus load in CSFRange Mean

HSV 1 42 (78%) acute encephalitisor meningitis12 (22%) no information

2.3 x 102 – 1.8 x 106 6.9 x 104

HSV 2 10 (77%) meningitis orencephalitis3 (23%) perinatal infection

3.9 x 103 – 4.9 x 105 5.5 x 104

5.4 x 104 – 9.9 x 109 1.5 x 106

SW Aberle and E Puchhammer-Stockl, Journal of Clinical Virology 25 (2002) S79-S85

Page 28: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False negatives and strain variation

• The ‘Swedish’ strain of C. trachomatis emerged in 2006

• Missing 377 base pairs of the cryptic plasmid

• Several counties in Sweden reported an apparent decrease in reported cases

• Variant scarcely reported outside of Sweden

Page 29: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

False negatives and Inhibition:Internal Control System: goals

• To identify false-negative results a specimen known to be positive for the target marker and misclassified by the device and/or test used, most likely due to inhibition and/or loss of the sample.

• To generate more accurately quantitative results by compensating for loss of material or inhibition (calibration - accuracy).

• Evaluate new isolation systems, equipment, disposables, reagents and change assays in a short period of time.

BUT• A limited number of publications uses internal controls together with the in-

house developed assays.

• From reports in QCMD proficiency programs only a limited number of laboratories are routinely using them.

Page 30: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Internal Control SystemNCCLS global consensus guideline MM6-A

• Incorporation of an internal calibrator in specimens allows for the control of the effect of inhibitors.

• The calibrator may be in the amplification reaction along with the target sequence, or run in parallel reactions.

• The control shall as far as possible be used throughout the whole process, i.e. extraction, amplification/hybridisation, detection.

CLSI MM6-A, Approved guidelines 2003

Page 31: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Amplification Reactions :Monitoring False Negatives Results

• detection of inhibitors by addition of • whole target organism added to test in duplicate• target nucleic acid added to test in duplicate• internal control (IC)

• homologous IC : target derived• heterologous IC: non target-derived

- extrinsic : added to sample- intrinsic : already present in sample

CLSI MM6-A, Approved guidelines 2003

Page 32: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

• Improvement of accuracy and precision of positive as well as non-detectable results.

• Valuable information on isolation procedures from clinical material, evaluation of new methods as well as improvements of existing technologies.

• Direct comparison of isolation efficiency combined with amplification, from different clinical material.

• Quality control of reagents and disposables used for isolation and amplification procedure.

A ‘universal’ approach to internal control

Phocine HerpesVirus 1 (PhHV)Herpesvirus

Phocine Distemper Virus (PDV)Morbillivirus

Page 33: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

• Use PhHV and PDV as representative of a DNA and RNA virus.

• Does increased input volume result in improved recovery?

• Output volume always identical, and concentration of internal control virus chosen to give identical Ct values in output volume.

Isolation of nucleic acids using different commercially available systems

Page 34: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Comparison isolation methods and volumes Ct values internal control PDV

200/100

MagnaPure

1000/100

ABI7700, 100 ul output

32,0

31,5

31,0

30,5

30,0

29,5

29,0

28,5

28,0

27,5

Ct v

alue

s on

AB

I770

0

P=0.469

Page 35: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Comparison isolation methods and volumes Ct values internal control PDV

200/100

MagnaPure

1000/100 200/100 850/100

AmpliPrep TNAI

ABI7700, 100 ul output

32,0

31,5

31,0

30,5

30,0

29,5

29,0

28,5

28,0

27,5

Ct v

alue

s on

AB

I770

0

P=0.469

P<0.0001

Page 36: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Comparison isolation methods and volumes Ct values internal control PDV

200/100

MagnaPure

1000/100 200/100 850/100

AmpliPrep TNAI

ABI7700, 100 ul output

32,0

31,5

31,0

30,5

30,0

29,5

29,0

28,5

28,0

27,5

Ct v

alue

s on

AB

I770

0

P=0.469

P<0.0001

Page 37: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Comparison isolation methods and volumes Ct values internal control PDV

200/100

MagnaPure

1000/100 200/100 850/100 200/100 1000/100

AmpliPrep TNAI DSP UltraSenseQiagen

ABI7700, 100 ul output

32,0

31,5

31,0

30,5

30,0

29,5

29,0

28,5

28,0

27,5

Ct v

alue

s on

AB

I770

0

P=0.469 P<0.0001

P<0.0001

Page 38: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Comparison isolation methods and volumesCt values internal control PhHV-1

ABI7700, 100 ul output

40

38

36

34

32

30

28

26

24

200/100

MagnaPure

1000/100 200/100 850/100 200/100 1000/100

AmpliPrep TNAI DSP UltraSenseQiagen

Ct v

alue

s on

AB

I770

0

P<0.0001

P=0.0118

P=0.0603

Page 39: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Internal Quality Control (IQC)

• Monitoring run to run performance.

• Monitoring reagents / kit batches, Evaluate new isolation systems, equipment, disposables, reagents.

• Often used in the validation of patient results / decision making process.

• Provides additional confidence in the results generated, both positive and negative (non-detected) ones.

30

32

34

36

38

40

42

0 10 20 30 40 50 60 70 80

Samplenumber

Ct v

alue

PhH

V1

12

16

20

24

28

32

36

40

44

0 100 200 300 400 500

Dilution Factor

Ct v

alue

PhH

V1

20ul in DNA 20ul in TNA

Page 40: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

External Quality Assessment (EQA / PT): What do we want to monitor?

• How well am I doing?

- Individual sample

- Across the QC panel / QC programmes

• Is my overall performance ‘acceptable’? Detects weak spots

• How did my performance this time compare with previous panels?

• How well do I compare with other participants using the same

technology? Improves reliability of results, gives confidence in

reporting results

• How do the technologies compare? Allows comparison of

performance with other laboratories & methods

Page 41: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

How can Proficiency Testing / EQA support laboratory quality?

• Monitor the performance of laboratory testing process

• Allow comparison to other laboratories

• Assist in identification of potential testing problems

• Performance of multiple labs, helps identify quality issues that were not noticed in individual labs

• Complements the clinical laboratory’s Quality Assurance System (regulatory requirements)

• cover clinically relevant targets and developing targets (including genetic variations)

• Educate participants in quality assurance issues

Page 42: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Under requirements of ISO 15189…

5.6.4: “ The laboratory shall participate in inter laboratory comparisons… . The laboratory shall monitor the results… . Inter laboratory comparison programs shall be in substantial agreement with ISO/IEC Guide 43-1.”

ISO 15189 (Medical laboratories – Particular requirements for quality and competence)

Page 43: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Molecular Diagnostic Tests : Proficiency Testing

• To confirm skill of lab in test performance• To ensure reproducibility• To validate amplification methods• Frequency :

- 2-3 testing events / year- 5 test samples / testing event covering full range : non reactive ®

highly reactive• Samples :

- whole organisms or isolated nucleic acids- previously characterized specimens, blinded and arranged in panel- or duplicate, blinded specimens (internal consistency)

NCCLS MM3-A. , 1995CLSI MM3-A2 Approved guidelines 2003

Page 44: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Interactivity

Page 45: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Number of QCMD programmes and registrations per year

0

5

10

15

20

25

30

35

40

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Num

ber o

f pro

gram

mes

Year

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Num

ber o

f pro

gram

me

regi

stra

tions

Year

RegistrationsAccumulative total

in 2008

- over 4500 datasets- over 1400 participants- 109 mean participant per program- > 35 full programs in 2008

Page 46: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

The Proposed program…..

Chlamydia

VZV

HIV-DNA

HIV-RNA

HSV 1 & 2

HCV

Legionella

EV

HBV

N. gonorrhoeae

HCV genotyping

Toxoplasma

Mycobacterium

EBV

CMV

HIV Drug Resistance

West Nile Virus

Influ BInflu A

B19

RSV

Mycoplasma

Bordetella

BK / JC

HPV

Nucleic Acid Extraction

…….

Page 47: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

What is the clinically relevant level of detection?

Clinical picture and herpesvirusDNA load in CSF

Virus Clinical picture Virus load in CSFRange Mean

HSV 1 42 (78%) acute encephalitisor meningitis12 (22%) no information

2.3 x 102 – 1.8 x 106 6.9 x 104

HSV 2 10 (77%) meningitis orencephalitis3 (23%) perinatal infection

3.9 x 103 – 4.9 x 105 5.5 x 104

5.4 x 104 – 9.9 x 109 1.5 x 106

SW Aberle and E Puchhammer-Stockl, Journal of Clinical Virology 25 (2002) S79-S85

Page 48: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

83

93

93

77

92

93

95

97

64

62

98

95

36

5

23

7

8

7

6

17

3

5

2

38

1

0% 20% 40% 60% 80% 100%

HSV03-12

HSV03-11

HSV03-10

HSV03-9

HSV03-8

HSV03-7

HSV03-6

HSV03-5

HSV03-4

HSV03-3

HSV03-2

HSV03-1

% Correct % Incorrect % Equivocal

7.6x106 copies/ml HSV 2

negative

6.6x102 copies/ml HSV 1

6.6x102 copies/ml HSV 1

1.1x107 copies/ml HSV 1

negative

VZV copies/ml

2.0x103 copies/ml HSV 2

2.4x102 copies/ml HSV 2

6.1x103 copies/ml HSV 1

4.8x103 copies/ml HSV 2

4.0x103 copies/ml HSV 1

HSV: Results per panel member

Focus on specific areas….

Page 49: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Focus on specific areas….

Sample Sample Status

0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3HSV07-07 Strong Positive 179 0 0 6 3 0 0 0 31 0 0 3 41 0 0 2 103 0 0 1 1 0 0 0HSV07-10 Strong Positive 179 0 0 6 3 0 0 0 32 0 0 2 41 0 0 2 102 0 0 2 1 0 0 0HSV07-02 Positive 179 0 6 0 3 0 0 0 31 0 3 0 41 0 2 0 103 0 1 0 1 0 0 0HSV07-08 Positive 177 0 8 0 3 0 0 0 30 0 4 0 40 0 3 0 103 0 1 0 1 0 0 0HSV07-04 Weak Positive 163 22 0 0 3 0 0 0 23 11 0 0 39 4 0 0 97 7 0 0 1 0 0 0HSV07-11 Strong Positive 178 0 0 7 3 0 0 0 33 0 0 1 40 0 0 3 101 0 0 3 1 0 0 0HSV07-01 Strong Positive 177 0 0 8 3 0 0 0 31 0 0 3 40 0 0 3 102 0 0 2 1 0 0 0HSV07-05 Positive 173 0 12 0 3 0 0 0 28 0 6 0 39 0 4 0 102 0 2 0 1 0 0 0HSV07-09 Weak Positive 111 74 0 0 2 1 0 0 13 21 0 0 26 17 0 0 69 35 0 0 1 0 0 0HSV07-12 Negative 171 0 0 14 3 0 0 0 33 0 0 1 40 0 0 3 94 0 0 10 1 0 0 0HSV07-03 Negative 175 0 0 10 3 0 0 0 31 0 0 3 40 0 0 3 100 0 0 4 1 0 0 0HSV07-06 Negative 180 0 0 5 2 0 0 1 34 0 0 0 42 0 0 1 101 0 0 3 1 0 0 0

PCR NASBA e

n=3 n=34 n=43

Conventional Real time

n=104 n=1

TotalAll technologies

n=185In-house Commercial In-houseCommercial

• Full summary reports and personalised individual reports

Page 50: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

50 copies/ml

negative

2 x105 copies/ml

200 copies/ml

2 x104 copies/ml

2x103 copies/ml

2 x105 copies/ml

1 x103 copies/ml89

99

100

100

66

99

99

35 64

11

1

33

1

1

1

1

0% 20% 40% 60% 80% 100%

HIV-8

HIV-7

HIV-6

HIV-5

HIV-4

HIV-3

HIV-2

HIV-1

% Correct % Incorrect % Equivocal

HIV-1 program 2003

Sensitivity

Page 51: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Forde, Templeton et al. 2004 : A multi-centre pilot proficiency

programme to assess the quality of molecular detection of

respiratory viruses.

Key Points

• For Rhinoviruses (panel D) Only one laboratory reported typing results.

• For Adenovirus (panel E) Only one laboratory submitted typing results.

• For panels A, B, and E mixed samples were considered correct if both types were reported.

• The only False positive result was reported on the rhinovirus (panel D)

• One laboratory did not submit results for the negative Adenovirus (panel E)

Dilution Series

Virus n= Hi Med Low Mixed sample

Negative sample

Panel A Influenza A* 17 94 65 59 Influenza B 17 100 100 65 88 100 Panel B PIV1 14 86 57 14 PIV3 14 93 50 29 86 100 Panel C RSV* 16 88 69 56 na 100 hMPV 14 93 93 86 na 100 Panel D Rhinovirus 16 11 100 91 45 Rhinovirus 72 11 73 73 36 Rhinovirus 90 11 100 91 45 100 91 Panel E Adenovirus 4 14 93 86 71 Adenovirus 7 14 86 57 57 100 93 Panel F Coronavirus 229E 8 100 75 50 50 100 Coronavirus OC43 8 100 86 0 57 100

Page 52: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

92.07%HBV2004

92.46%HBV2003

HBV04-15HBV01Samples→Progyears ↓

Performance in quantitative tests; reproducibility in QC programmes

• ForHBV the overall performance of all participants/technologies in both years was not significantly different

• Variation was < 0.4 log 10

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

2006 2007 2008

Log10 Co

pies/m

l

Year

HSV 2

Page 53: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Q-Bot

EQA Use Only

EQA providers demand EQA panel is analysed in same manner as clinical specimen.

Pressure to perform, consequences of failure, potential loss of licence (in some countries).

Puts different emphasis on EQA sample testing & goalof the EQA programme

What Do you Do?

Best Laboratory TechnicianEQA panel tested separately.Extra special treatment GOOD PRACTICE ?

How to use EQA panels?

Page 54: Quality Assurance in PCR testing - WIV-ISP · Practical application in detecting & identifying infectious agents (unculturable organisms). Identification & characterisation of antimicrobial

Quality assurance in molecular diagnostics: Conclusions

• Rapidly moving field, new technologies, new assays, high expectations- In some cases the technology still has to prove itself in the clinical setting

• The introduction of new systems both for nucleic acid isolation as well as detection, as well new reagents and disposables should be validated. In general, data are not provided, limited or lacking.

• Internal quality assessment improves molecular diagnostic methods.

• The introduction of a (universal) internal control enables to implement molecular diagnostic assays with a higher confidence. And is a must.

• What is appropriate for one lab may not be for another.• Education in quality tools is essential