Comparability ofLaboratory Tests - James O. Westgard

51
Comparability of Comparability of Laboratory Tests Laboratory Tests James O. Westgard James O. Westgard University of Wisconsin Medical School University of Wisconsin Medical School Westgard QC, Inc. Westgard QC, Inc. Madison, WI Madison, WI www.westgard.com www.westgard.com 1 1

description

Comparability of Laboratory Tests James O. Westgard - University of Wisconsin Medical School Westgard QC, Inc.Madison, WI

Transcript of Comparability ofLaboratory Tests - James O. Westgard

Page 1: Comparability ofLaboratory Tests - James O. Westgard

Comparability of Comparability of Laboratory TestsLaboratory Tests

James O. WestgardJames O. WestgardUniversity of Wisconsin Medical SchoolUniversity of Wisconsin Medical School

Westgard QC, Inc.Westgard QC, Inc.Madison, WIMadison, WI

www.westgard.comwww.westgard.com

11

Page 2: Comparability ofLaboratory Tests - James O. Westgard

““A rose is a rose is a rose!A rose is a rose is a rose!””

22

Page 3: Comparability ofLaboratory Tests - James O. Westgard

But, is a But, is a ““glucose a glucose a glucose a glucose a glucose?glucose?””

Or, is a glucose a BGM glucose Or, is a glucose a BGM glucose or an eor an e--glucose from HbA1c?glucose from HbA1c?

Or, is a GHb a HbA1c or an Or, is a GHb a HbA1c or an

NGSP certified HbA1c?NGSP certified HbA1c?33

Page 4: Comparability ofLaboratory Tests - James O. Westgard

Harmonization is a major issue Harmonization is a major issue for medical laboratories today!for medical laboratories today!

““Improving Clinical Laboratory Testing Through Improving Clinical Laboratory Testing Through Harmonization: An International ForumHarmonization: An International Forum””, Oct 26, Oct 26--

27, 2010, NIST, Gaithersburg MD27, 2010, NIST, Gaithersburg MD

““The AACC is hosting an The AACC is hosting an invitation onlyinvitation only conference conference on global harmonization of results from clinical on global harmonization of results from clinical laboratory testing procedures for which no reference laboratory testing procedures for which no reference measurement procedure exists or is likely to be measurement procedure exists or is likely to be developed. The twodeveloped. The two--day conference will seek to day conference will seek to strengthen quality of laboratory measurements and strengthen quality of laboratory measurements and improve patient care by developing consensus on improve patient care by developing consensus on technical and organizational processes to achieve technical and organizational processes to achieve harmonization of clinical laboratory results. harmonization of clinical laboratory results.

44

Page 5: Comparability ofLaboratory Tests - James O. Westgard

HarmonizationHarmonization

CLSI definition: CLSI definition: harmonization harmonization in glycohemoglobin (GHB) testing, the in glycohemoglobin (GHB) testing, the process by which GHB test results among process by which GHB test results among laboratories are made comparable to a laboratories are made comparable to a common reference. common reference.

Harmonization is a process!Harmonization is a process!

Comparability is a measure of the outcome!Comparability is a measure of the outcome!

Are HbA1c test results comparable today?Are HbA1c test results comparable today?

55

Page 6: Comparability ofLaboratory Tests - James O. Westgard

Need for Comparability of Need for Comparability of Medical Laboratory TestsMedical Laboratory Tests

Patients being treated in a healthcare Patients being treated in a healthcare organization that includes doctors offices & organization that includes doctors offices & clinics & hospitals, each having their own clinics & hospitals, each having their own laboratory testing servicelaboratory testing service

Patients being treated in multiple healthcare Patients being treated in multiple healthcare organizations organizations

In a geographic regionIn a geographic region

Throughout a countryThroughout a country

Throughout the worldThroughout the world66

Page 7: Comparability ofLaboratory Tests - James O. Westgard

Need for ComparabilityNeed for Comparability

Electronic Medical RecordElectronic Medical Record

Merging patient information from many Merging patient information from many different sources, including different different sources, including different laboratorieslaboratories

Assumes that Assumes that ““A glucose is a glucose is a A glucose is a glucose is a glucoseglucose””, but we know that isn, but we know that isn’’t true!t true!

Blood glucose meter results from patientBlood glucose meter results from patient

PointPoint--ofof--Care results from Doctors OfficesCare results from Doctors Offices

Hospital results from medical laboratoriesHospital results from medical laboratories

77

Page 8: Comparability ofLaboratory Tests - James O. Westgard

Need for ComparabilityNeed for Comparability

New New ““PayPay--forfor--PerformancePerformance””

incentives for incentives for quality improvementquality improvement

Chronic diseases of high priorityChronic diseases of high priority

Likely to utilize laboratory tests as measures Likely to utilize laboratory tests as measures of performanceof performance

Treatment goals may assume absolute Treatment goals may assume absolute measures that are comparable from method measures that are comparable from method to method, lab to lab, and over timeto method, lab to lab, and over time

88

Page 9: Comparability ofLaboratory Tests - James O. Westgard

How assure comparability?How assure comparability?

TraceabilityTraceability

In principle, laboratory methods should have In principle, laboratory methods should have common reference materials and methods to common reference materials and methods to establish the correct or true test valuesestablish the correct or true test values

Laboratory Quality ManagementLaboratory Quality Management

Quality systems, method selection, method Quality systems, method selection, method validation, QC, calibrators, reagent lots etc.validation, QC, calibrators, reagent lots etc.

External Quality AssessmentExternal Quality Assessment

In practice, test performance must be In practice, test performance must be monitored to demonstrate comparability monitored to demonstrate comparability

99

Page 10: Comparability ofLaboratory Tests - James O. Westgard

How assure comparability?

1010

BQC3Ch 18P 281

(3) Validate MethodPerformance (CV,bias)

(1) Define Goals for Intended Use

(TEa, Dint)

(3a) Manufacturer’sClaims

(1a) Regulatory &AccreditationRequirements

(1b) Clinical and Medical Applications

(2) Select AnalyticMeasurement

Procedure

(2a) Traceability& Calibration

(2b) Manufacturer’sReference Methods

& Materials

(4) Design SQC(rules, N, F)

(5) FormulateAQC Strategy

(11) Improve AQCEffectiveness

(5b) Lab Evaluationof Residual Risk

(5a) Manufacturer’sRisk Analysis

(6) Develop AQC Plan

(10) Monitor AQCEffectiveness (f), EQA

(7) Implement AQC System

(6a) QC Toolbox

(8) Verify Attainmentof Intended Quality

of Test Results

(9) Measure Quality& Uncertainty

Page 11: Comparability ofLaboratory Tests - James O. Westgard

Traceability Traceability ––

not a new conceptnot a new concept

Guidelines from the Guild of Soup MakersGuidelines from the Guild of Soup Makers

““Every new master sees to it that the guild weights Every new master sees to it that the guild weights and measures are renewed on his appointment. The and measures are renewed on his appointment. The old ones are then stored for comparison. old ones are then stored for comparison.

““If the measures are the same, the ingredients must If the measures are the same, the ingredients must also be the same. Our soups do not merely conform also be the same. Our soups do not merely conform with the standards, they are part of the standards.with the standards, they are part of the standards.

““An unbroken line that comes down to us from the An unbroken line that comes down to us from the days of the conquest.days of the conquest.””

Jason Goodwin. The Janissary Tree. Pickador Books, Jason Goodwin. The Janissary Tree. Pickador Books, NY, p 38. NY, p 38. A mystery story set in the 1800s in the A mystery story set in the 1800s in the Ottoman Empire featuring Investigator Yashim.Ottoman Empire featuring Investigator Yashim.

1111

Page 12: Comparability ofLaboratory Tests - James O. Westgard

Metrological Standards for Metrological Standards for Quality Measurement ProcessesQuality Measurement Processes

TraceabilityTraceability

Property of the results of a measurement or the value Property of the results of a measurement or the value of a standard whereby it can be related to stated of a standard whereby it can be related to stated references, usually national or international references, usually national or international standards, through an standards, through an unbroken chain of unbroken chain of comparisons comparisons all having stated uncertaintiesall having stated uncertainties

Uncertainty of measurementUncertainty of measurement

Parameter, associated with the result of a Parameter, associated with the result of a measurement, that characterizes the dispersion of the measurement, that characterizes the dispersion of the values that could reasonably be attributed to the values that could reasonably be attributed to the measurandmeasurand

1212

Page 13: Comparability ofLaboratory Tests - James O. Westgard

Traceability ChainTraceability Chain

1313

Page 14: Comparability ofLaboratory Tests - James O. Westgard

Issues with TraceabilityIssues with Traceability

Definition of analyte and unitsDefinition of analyte and units

““PhysicoPhysico--chemical complexitychemical complexity””

of biologic of biologic measurands and matraciesmeasurands and matracies

Lack of Primary Reference MaterialsLack of Primary Reference Materials

Lack of Primary Reference MethodsLack of Primary Reference Methods

Lack of Secondary CalibratorsLack of Secondary Calibrators

Lack of Secondary Reference MethodsLack of Secondary Reference Methods

Need for Reference Laboratory ServicesNeed for Reference Laboratory Services1414

Page 15: Comparability ofLaboratory Tests - James O. Westgard

““Medical TraceabilityMedical Traceability””

Thienpont et al. Clin Chem Lab Med Thienpont et al. Clin Chem Lab Med 2004;42:8422004;42:842--850850

““PhysioPhysio--chemical complexitychemical complexity””

Complex biologic matricesComplex biologic matrices

Analyte may be a class of substancesAnalyte may be a class of substances

Lack of specificity in measurement proceduresLack of specificity in measurement procedures

Need for traceability models that utilize Need for traceability models that utilize ““acceptedaccepted””

reference materials and methods, reference materials and methods,

valuevalue--assignment protocols, laboratory assignment protocols, laboratory networks as base of referencenetworks as base of reference

1515

Page 16: Comparability ofLaboratory Tests - James O. Westgard

Model II: International Reference Model II: International Reference Method and CalibratorMethod and Calibrator

1616

Page 17: Comparability ofLaboratory Tests - James O. Westgard

““Medical TraceabilityMedical Traceability””

of HbA1cof HbA1cIFCC Reference

Method

Value-assignment of Calibrator Materials National Reference

Methods (US NGSP)NGSP Certification

Program ManufacturersMethods

Manufacturers Calibrators Laboratory

MethodsTest

Samples

Test Results

CAPPT

IFCC LabNetwork

1717

Page 18: Comparability ofLaboratory Tests - James O. Westgard

Comparability in the Real WorldComparability in the Real World HbA1c exampleHbA1c example

LentersLenters--Westra E, Slingerland RJ. Six of Westra E, Slingerland RJ. Six of Eight Hemoglobin A1c PointEight Hemoglobin A1c Point--ofof--Care Care Instruments Do Not Meet the General Instruments Do Not Meet the General Accepted Analytical Performance Accepted Analytical Performance Criteria. Clin Chem 2010;56:44Criteria. Clin Chem 2010;56:44--52.52.

Bruns DE, Boyd JC. Few PointBruns DE, Boyd JC. Few Point--ofof--Care Care Hemoglobin A1c Assay Methods Meet Hemoglobin A1c Assay Methods Meet Clinical Needs. Clin Chem 2010;56:4Clinical Needs. Clin Chem 2010;56:4--6.6.

1818

Page 19: Comparability ofLaboratory Tests - James O. Westgard

POC POC ““Waived TestWaived Test””

FDA review requires manufacturer to FDA review requires manufacturer to define Allowable Total Error to validate define Allowable Total Error to validate quality of quality of ““waivedwaived””

teststests

Recommends use of Recommends use of ““error gridserror grids””

Clarke et al. Diabetes Care 1987;10:622Clarke et al. Diabetes Care 1987;10:622--88

Parkes et al. Diabetes Care 2000;23:1143Parkes et al. Diabetes Care 2000;23:1143--88

CLSI EP27P published late 2009CLSI EP27P published late 2009

““How to Construct and Interpret an Error Grid How to Construct and Interpret an Error Grid for Diagnostic Assaysfor Diagnostic Assays””

1919

Page 20: Comparability ofLaboratory Tests - James O. Westgard

FDA Guidance (2008)FDA Guidance (2008) Allowable Total Error GridAllowable Total Error Grid

Guidance for Industry and FDA Staff: Recommendations for Clinical LaboratoryImprovement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers’of Invitro Diagnostic Devices. Jan 30, 2008, Food and Drug Adminstration 2020

Page 21: Comparability ofLaboratory Tests - James O. Westgard

Calculation of Calculation of ““estimatedestimated--GlucoseGlucose””

ADA recommends conversion of HbA1c ADA recommends conversion of HbA1c results to results to ““estimated glucoseestimated glucose””

to help to help

patients understand the clinical implicationspatients understand the clinical implications

ISO standard for Blood Glucose MetersISO standard for Blood Glucose Meters

20% TEa or 15 mg/dL < 75 mg/dL20% TEa or 15 mg/dL < 75 mg/dL

CLIA criterion for acceptable performanceCLIA criterion for acceptable performance

6 mg/dL or 10%, whichever is larger6 mg/dL or 10%, whichever is larger

HbA1c Requirements forHbA1c Requirements for Monitoring Glycemic ControlMonitoring Glycemic Control

2121

Page 22: Comparability ofLaboratory Tests - James O. Westgard

“e-Glucose”

Requirements ISO 15197 vs CLIA

2222

Page 23: Comparability ofLaboratory Tests - James O. Westgard

Quality Quality ‘‘StandardsStandards’’

for BMG for BMG Devices may change! 15%???Devices may change! 15%???

2323

Page 24: Comparability ofLaboratory Tests - James O. Westgard

NGSP certification requirementNGSP certification requirement

95% agreement within specified limits95% agreement within specified limits

±±

0.85 %Hb in 20090.85 %Hb in 2009

±±

0.75 %Hb in 20100.75 %Hb in 2010

CAP proficiency testing requirementCAP proficiency testing requirement

TEa of 10% in 2009TEa of 10% in 2009

TEa of 8.0% in 2010TEa of 8.0% in 2010

TEa of 6.0% in 2011TEa of 6.0% in 2011

NGSP and CAP CriteriaNGSP and CAP Criteria

2424

Page 25: Comparability ofLaboratory Tests - James O. Westgard

2010 NGSP vs CAP Criteria 2010 NGSP vs CAP Criteria WhatWhat’’s wrong with this picture?s wrong with this picture?

2525

Page 26: Comparability ofLaboratory Tests - James O. Westgard

HbA1c Requirement for HbA1c Requirement for Diagnosis of DiabetesDiagnosis of Diabetes

ADA recommends cutoff of 6.5 %HbADA recommends cutoff of 6.5 %Hb

Values from 5.7 to 6.4 %Hb higher riskValues from 5.7 to 6.4 %Hb higher risk

Values of 5.6 %Hb and less Values of 5.6 %Hb and less ““normalnormal””

Interpretation for error gridInterpretation for error grid

False NegativesFalse Negatives: Values of 6.5 %Hb that : Values of 6.5 %Hb that are erroneously measured as 5.6 %Hb or lessare erroneously measured as 5.6 %Hb or less

False PositivesFalse Positives: Values of 5.6 %Hb that are : Values of 5.6 %Hb that are erroneously measured as 6.5 %Hb or highererroneously measured as 6.5 %Hb or higher

2626

Page 27: Comparability ofLaboratory Tests - James O. Westgard

ADA Diagnostic Criterion vs NGSP vs CAP

2727

Page 28: Comparability ofLaboratory Tests - James O. Westgard

ADA treatment goal 7.0 %HbADA treatment goal 7.0 %Hb

When reWhen re--evaluate or change treatment?evaluate or change treatment?

Earlier ADA recommendation was to Earlier ADA recommendation was to adjust treatment if HbA1c 8.0 %Hb or adjust treatment if HbA1c 8.0 %Hb or greatergreater

Change of 1.0 %Hb is equivalent to a Change of 1.0 %Hb is equivalent to a monitoring requirement of about 14%monitoring requirement of about 14%

Absent from latest guidelinesAbsent from latest guidelines

HbA1c Requirement for HbA1c Requirement for Monitoring TreatmentMonitoring Treatment

2828

Page 29: Comparability ofLaboratory Tests - James O. Westgard

Diagnosis & Monitoring (Δ1.0%Hb) vs NGSP vs CAP PT

2929

Page 30: Comparability ofLaboratory Tests - James O. Westgard

What quality is needed in a What quality is needed in a medical laboratory for the medical laboratory for the

intended clinical use of HbA1c?intended clinical use of HbA1c?

CAP criterion was 10% in 2009, 8% in CAP criterion was 10% in 2009, 8% in 2010 (8%) and will be 6% in 20112010 (8%) and will be 6% in 2011

NGSP 2010 criterion of NGSP 2010 criterion of ±±0.75 %Hb 0.75 %Hb corresponds to TEa of 10.7% @ 7.0 %Hb;corresponds to TEa of 10.7% @ 7.0 %Hb;

CLIA criterion for acceptable performance CLIA criterion for acceptable performance of glucose is 10%of glucose is 10%

3030

Page 31: Comparability ofLaboratory Tests - James O. Westgard

Example ApplicationExample ApplicationMethod Performance Characteristics and QC Design

Test (Units) Glycated Hemoglobin (%Hb)Method (Analyzer) DCA Vantage

Medical Decision LevelsConcentrations

LOW Xc MID Xc High Xc5.0 7.0 9.0

CLIA Quality Criterion %TEa 10% 10% 10%

Precision (Replication or QC Data)SD

Mean%CV

Bias (Comparison, PT, Peer Data)Calculated Bias

%Bias

Sigma‐Metric(%TEa)/%CV

(%TEa ‐

%Bias)/%CV

SQC from Sigma tool, Control RulesTotal Number Measurements, NAnalytical QC Strategy

3131

Page 32: Comparability ofLaboratory Tests - James O. Westgard

Precision results Precision results from Lenters Studyfrom Lenters Study

Lenters-Westra E, Slingerland RJ. Six of Eight Hemoglobin A1c Point-of-Care Instruments Do Not Meet the General Accepted Analytical Performance Criteria. Clin Chem 2010;56:44-52.

3232

Page 33: Comparability ofLaboratory Tests - James O. Westgard

Example ApplicationExample ApplicationMethod Performance Characteristics and QC Design

Test (Units) Glycated Hemoglobin (%Hb)Method (Analyzer) DCA Vantage

Medical Decision LevelsConcentrations

LOW Xc MID Xc High Xc5.0 7.0 9.0

CLIA Quality Criterion %TEa 10% 10% 10%

Precision (Replication or QC Data)SD

Mean 5.1%Hb 11.2%Hb%CV 1.8% 3.7%

Bias (Comparison, PT, Peer Data)Calculated Bias

%Bias

Sigma‐Metric(%TEa)/%CV

(%TEa ‐

%Bias)/%CV

SQC from Sigma tool, Control RulesTotal Number Measurements, NAnalytical QC Strategy

3333

Page 34: Comparability ofLaboratory Tests - James O. Westgard

Accuracy results Accuracy results --

Comparison Comparison with avg of 3 reference methodswith avg of 3 reference methods

3434

Page 35: Comparability ofLaboratory Tests - James O. Westgard

What do the statistics tell us What do the statistics tell us about the size of errors?about the size of errors?

For Xc=7.0 %HbFor Xc=7.0 %Hb

Yc = 0.97*7.0 + 0.12 = 6.91Yc = 0.97*7.0 + 0.12 = 6.91

SE = 6.91SE = 6.91––7.0 = 7.0 = --0.09 %Hb or 1.3%Bias 0.09 %Hb or 1.3%Bias

Yc = 1.03*7.0 Yc = 1.03*7.0 ––

0.0 = 7.210.0 = 7.21

SE = 7.21 SE = 7.21 ––

7.0 = +0.21 %Hb or 3.0%Bias7.0 = +0.21 %Hb or 3.0%Bias

3535

Page 36: Comparability ofLaboratory Tests - James O. Westgard

Example ApplicationExample ApplicationMethod Performance Characteristics and QC Design

Test (Units) Glycated Hemoglobin (%Hb)Method (Analyzer) DCA Vantage

Medical Decision LevelsConcentrations

LOW Xc MID Xc High Xc5.0 7.0 9.0

CLIA Quality Criterion %TEa 10% 10% 10%

Precision (Replication or QC Data)SD

Mean 5.1%Hb 11.2%Hb%CV 1.8% 3.7%

Bias (Comparison, PT, Peer Data)Calculated Bias

%Bias 1.3‐3.0%

Sigma‐Metric(%TEa)/%CV

(%TEa ‐

%Bias)/%CV

SQC from Sigma tool, Control RulesTotal Number Measurements, NAnalytical QC Strategy

3636

Page 37: Comparability ofLaboratory Tests - James O. Westgard

WhatWhat’’s sigma?s sigma?

Estimate precision midrange as average of Estimate precision midrange as average of observed CVs at 5.1 and 11 %Hbobserved CVs at 5.1 and 11 %Hb

CV = ~ (1.8+3.7)/2 = ~ 2.75%CV = ~ (1.8+3.7)/2 = ~ 2.75%

Calculate sigma for each lot #Calculate sigma for each lot #

Sigma = (%TEaSigma = (%TEa--%Bias)/(%CV)%Bias)/(%CV)

Sigma = (10%Sigma = (10%--1.3%)/2.75% = 3.161.3%)/2.75% = 3.16

Sigma = (10%Sigma = (10%--3.0%)/2.75% = 2.553.0%)/2.75% = 2.55

3737

Page 38: Comparability ofLaboratory Tests - James O. Westgard

What QC is needed? What QC is needed? 3 Levels Control (2.55 Sigma)3 Levels Control (2.55 Sigma)

3838

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0 1.0 2.0 3.0 4.0

1.65 2.65 3.65 4.65 5.65

13s/2of32s/R4s/31s/6x 0.07 ----- 6 113s/2of32s/R4s/31s/6x 0.03 ----- 3 213s/2of32s/R4s/31s 0.05 ----- 6 113s/2of32s/R4s/31s 0.02 ----- 3 113s 0.01 ----- 6 112.5s 0.03 ----- 3 113s 0.01 ----- 3 113s 0.00 ----- 1 1

Pfr Ped N R

Pro

babi

lity

for

Rej

ecti

on (

P)

Systematic Error (SE, multiples of s)

Sigma Scale3 4 5

Page 39: Comparability ofLaboratory Tests - James O. Westgard

What QC is needed? What QC is needed? 3 Levels Control (3.16 Sigma)3 Levels Control (3.16 Sigma)

3939

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0 1.0 2.0 3.0 4.0

1.65 2.65 3.65 4.65 5.65

13s/2of32s/R4s/31s/6x 0.07 ----- 6 113s/2of32s/R4s/31s/6x 0.03 ----- 3 213s/2of32s/R4s/31s 0.05 ----- 6 113s/2of32s/R4s/31s 0.02 ----- 3 113s 0.01 ----- 6 112.5s 0.03 ----- 3 113s 0.01 ----- 3 113s 0.00 ----- 1 1

Pfr Ped N R

Pro

babi

lity

for

Rej

ecti

on (

P)

Systematic Error (SE, multiples of s)

Sigma Scale3 4 5

Page 40: Comparability ofLaboratory Tests - James O. Westgard

Example ApplicationExample ApplicationMethod Performance Characteristics and QC Design

Test (Units) Glycated Hemoglobin (%Hb)Method (Analyzer) DCA Vantage

Medical Decision LevelsConcentrations

LOW Xc MID Xc High Xc5.0 7.0 9.0

CLIA Quality Criterion %TEa 10% 10% 10%

Precision (Replication or QC Data)SD

Mean 5.1%Hb 11.2%Hb%CV 1.8% ~2.75% 3.7%

Bias (Comparison, PT, Peer Data)Calculated Bias

%Bias 1.3‐3.0%

Sigma‐Metric(%TEa)/%CV

(%TEa ‐

%Bias)/%CV 3.16‐2.55SQC from Sigma tool, Control Rules Multi‐rule QCTotal Number Measurements, N All the QC you can afford!Analytical QC Strategy Hope and pray nothing goes wrong!

4040

Page 41: Comparability ofLaboratory Tests - James O. Westgard

Houston Houston ––

We have a problem!We have a problem! Or maybe not???Or maybe not???

Waived test doesnWaived test doesn’’t have to meet US t have to meet US CLIA minimum QC of 2/day, nor EQC of CLIA minimum QC of 2/day, nor EQC of 2/week or 2/month!2/week or 2/month!

Just follow manufacturerJust follow manufacturer’’s instructions s instructions and recommendationsand recommendations

Not required to validate method Not required to validate method performance!performance!

Not required to participate in PT!Not required to participate in PT!4141

Page 42: Comparability ofLaboratory Tests - James O. Westgard

WhatWhat’’s the point?s the point?

11stst

issue is whether the device can achieve issue is whether the device can achieve

the necessary quality when it is working the necessary quality when it is working correctly?correctly?

Method validation is critical, but not required!Method validation is critical, but not required!

Quality is controlled by FDA when device is Quality is controlled by FDA when device is approved as approved as ““waivedwaived””

22ndnd

issue is that QC only monitors the issue is that QC only monitors the

stable performance achieved stable performance achieved

Of little use if device canOf little use if device can’’t achieve desired t achieve desired performanceperformance

4242

Page 43: Comparability ofLaboratory Tests - James O. Westgard

Is quality acceptable for POC Is quality acceptable for POC application?application?

POC application POC application lessless demanding because demanding because the test should be used only for the test should be used only for monitoring, not diagnosis!monitoring, not diagnosis!

ADA guidelines recommend against using POC ADA guidelines recommend against using POC devices for diagnosis of diabetesdevices for diagnosis of diabetes

For monitoring therapy, what changes need to For monitoring therapy, what changes need to be detected? be detected?

See discussions on See discussions on www.westgard.comwww.westgard.com4343

Page 44: Comparability ofLaboratory Tests - James O. Westgard

WhatWhat’’s the problem?s the problem?

Huge effort needed to harmonize Huge effort needed to harmonize laboratory tests when have many different laboratory tests when have many different measurement principles and proceduresmeasurement principles and procedures

HbA1c illustrates the complexity, as well as HbA1c illustrates the complexity, as well as the difficulty in achieving comparabilitythe difficulty in achieving comparability

Certification of equivalent results only Certification of equivalent results only indicates comparability indicates comparability ““on average,on average,””

not that not that

individual patient results will be comparableindividual patient results will be comparable

““ProprietaryProprietary””

methodology limits efforts for methodology limits efforts for standardizing measurement proceduresstandardizing measurement procedures

4444

Page 45: Comparability ofLaboratory Tests - James O. Westgard

CAP Survey CAP Survey 20092009

Method Subgroup4545

Page 46: Comparability ofLaboratory Tests - James O. Westgard

Method # Labs Mean %CV1 50 6.73 4.52 27 6.45 5.13 87 6.49 3.84 269 6.43 3.65 209 6.74 2.96 139 6.81 2.87 147 6.76 2.99 25 6.46 4.410 133 6.70 3.011 38 6.87 3.612 139 6.78 3.213 15 6.61 3.514 21 6.41 7.615 26 6.79 3.816 106 6.81 2.517 97 6.78 2.818 25 6.62 4.619 349 6.67 3.320 36 6.84 6.521 141 6.60 3.322 71 6.87 3.123 248 6.80 1.924 84 6.80 1.725 27 6.66 3.426 124 6.80 4.1

TOT/AVG 2633 6.69 3.68%

Bias0.13‐0.15‐0.11‐0.170.140.210.16‐0.140.100.270.180.01‐0.190.190.210.180.020.070.240.000.270.200.200.060.20

0.09

Ab%Bias1.93%2.33%1.69%2.64%2.08%3.08%2.37%2.17%1.49%3.93%2.65%0.15%2.96%2.80%3.08%2.65%0.30%1.05%3.51%0.00%3.93%2.94%2.94%0.90%2.94%

2.26%

CAP 2009 Survey (NGSP Website 12/09)CAP 2009 Survey (NGSP Website 12/09)Sigma 10 Sigma8 Sigma61.79 1.35 0.901.50 1.11 0.722.19 1.66 1.132.04 1.49 0.932.73 2.04 1.352.47 1.76 1.042.63 1.94 1.251.78 1.33 0.872.84 2.17 1.501.69 1.13 0.572.30 1.67 1.052.81 2.24 1.670.93 0.66 0.401.90 1.37 0.842.77 1.97 1.17

2.62 1.91 1.192.11 1.67 1.242.71 2.11 1.501.00 0.69 0.383.03 2.42 1.821.96 1.31 0.673.72 2.66 1.614.15 2.98 1.802.68 2.09 1.501.72 1.23 0.75

2.32 1.72 1.114646

Page 47: Comparability ofLaboratory Tests - James O. Westgard

WhatWhat’’s the point?s the point?

Global and national efforts to harmonize Global and national efforts to harmonize HbA1c testing have improved quality, HbA1c testing have improved quality, BUT BUT biases still exist between different biases still exist between different measurement principles and procedures.measurement principles and procedures.

Still have problems with practices for Still have problems with practices for defining clinical treatment guidelines and defining clinical treatment guidelines and setting method operating specifications setting method operating specifications that can assure attainment of intended that can assure attainment of intended quality of test results.quality of test results.

4747

Page 48: Comparability ofLaboratory Tests - James O. Westgard

Further Complications Today! ISO Measurement Uncertainty

If measurement uncertainty is to be the measure of quality, the assumption is that all biases can be eliminated or corrected!

Not true for today’s proprietary methods!

Must “harmonize”

metrological theory with practical tools and techniques from current error management framework!

Medical laboratories should place priority on traceability, not measurement uncertainty!

4848

Page 49: Comparability ofLaboratory Tests - James O. Westgard

How explain this to customers who How explain this to customers who believe all methods give same results?believe all methods give same results?

4949

Page 50: Comparability ofLaboratory Tests - James O. Westgard

In conclusion, a hydrangea is a In conclusion, a hydrangea is a hydrangea is a hydrangea! hydrangea is a hydrangea!

5050

Page 51: Comparability ofLaboratory Tests - James O. Westgard

Harmonized, but not always the Harmonized, but not always the same!same!

5151