Laboratory Method Verification

Ola H. ElgaddarMD, PhD, MBA, CPHQ, LSSGB,Lecturer of Chemical Pathology

Alexandria UniversityOla.elgaddar@alexu.edu.eg

Quality

Doing the right thing right, from the first time and every time!

Accreditation

An institution or a program meets standards of quality set forth by an accrediting agency

Validation / Verification??Method validation: (Manufacturer concern)

Establishing the performance of a newdiagnostic tool.Confirmation, through the provision ofobjective evidence, that the requirements fora specific intended use or application havebeen fulfilled’ (doing correct test)......ISO 9001:2005

Validation / Verification??Method verification: (Lab / user concern)

A process to determine performancecharacteristics before a test system is utilizedfor patient testing.

Confirmation, through the provision ofobjective evidence, that specifiedrequirements have been fulfilled’ (doing testcorrectly)……ISO 9001:2005

According to WestgardØ The inner, hidden, deeper, secret meaningof method validation is error assessment.ØHow much error might be present in thetest result within your laboratory ?ØCould this degree of error affect theinterpretation and possibly patient care ?

If the potential error is large enough to lead tomisinterpretation, then the method is notacceptable.

Ø Random error, RE, or imprecision isdescribed as an error that can be eitherpositive or negative, whose direction andexact magnitude cannot be predicted, wherethe distribution of results when replicatemeasurements are made on a singlespecimen.ØUsually, due to error in Pipetting

Ø Systematic error, SE, or inaccuracy is anerror that is always in one direction,displacing the mean of the distribution fromits original value.ØIn contrast to random errors, systematicerrors are in one direction and cause all thetest results to be either high or low.ØEither constant or proportionateØUsually, due to error in calibration

Internal Quality Control (IQC) is used, on dailybasis, in the decision to accept or rejectresults of patients samples and enables thelab to describe and monitor the quality of itswork.-Usually it has 2 levels (Sometimes 3);representing the “Normal” and the“Pathological” analyte level.- Judged according to Westgard Multi-QCrules

External Quality Control (EQC) =Proficiency test is used, on monthly or Bi-weekly (Or others) basis, where labs from allover the world join the program and send theirused Method / Analyzer.- A statistical comparison is made and eachlab result is compared to the result of its peergroup in each analyte.

The following items need verification

ØAnalytical Specificity: Interference studiesØAnalytical Sensitivity: Calibration curve

Detection limitØReportable range: Linearity experimentØPrecision: Replication studyØAccuracy: Bias / Recovery studyØReference Intervals

Analytical SpecificityThe ability of an analytical method to detect

“ONLY” the analyte of interest.

Freedom from interference by any element or compound other than the analyte of interest

Analytical Specificity- Analytical Specificity is verified using

interference studies.- A validated method, known to be free of

the interfering substance is used. A seriesof samples containing increasedconcentrations of the interfering substanceare analyzed using that method, and themethod under study, then both results arecompared

Some Automated systems have a “HIL” index

Analytical Sensitivity- The ability of an analytical method todetect a low concentration of a givensubstance in a biological sample. The lowerthe detectable concentration, the greaterthe analytical sensitivity.- Detection limits studies

Analytical SensitivityOR,-The ability of an analytical method todetect (respond to) a change inconcentration of the analyte. The smallerthe detectable change (the change inconcentration that can result in a definitechange in the reported signal), the greateris the analytical sensitivity.- Slope of the Calibration Curve

REMEMBER!!

STANDARD

• It is a solution of known concentration.• Formed by dissolving a known amount of

an analyte in a specific volume of anaqueous solvent.

• Ex: Dissolving 100 mg glucose in 100 mlD.W gives a standard of a 100 mg / dlconcentration

CALIBRATOR

• A solution or a device of known quantitative or qualitative characteristics

(eg: concentration, activity, intensity)• Used to calibrate or adjust a measurement

procedure.• Matrix is preserved.

• The calibrator material is reconstituted &introduced to the analyzer before using anew lot of reagent, and its concentration isassigned.

• The calibrator is treated like samples andthe absorbance of the developed color isdetermined.

Using the provided concentration, the analyzer constructs a calibration curve

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 50 100 150 200 250

• If the given concentration is 100 mg / dl,the absorbance of this point is determined.

• By providing the analyzer with the rangewhere this method is linear, the calibrationcurve is extended so that any sampleconcentration lying within that linearityrange can be deduced from this curve

Detection Limits1.Lower Limit of Detection (LLD)= Limit of

the blank (LOB)

2.Biologic Limit of Detection (BLD)= Limit of Detection (LOD)

3.Functional Sensitivity (FS)= Limit of Quantification (LOQ)

Detection Limits1.Lower Limit of Detection (LLD)= Limit

of the blank (LOB)Most analytical instruments produce a signal evenwhen a blank (reagent without analyte) isanalyzed. This signal is referred to as the noiselevel. The LLD is the analyte concentration that isrequired to produce a signal greater than two (orthree) times the standard deviation of the noiselevel.

Detection Limits1.Lower Limit of Detection (LLD)= Limit

of the blank (LOB)Limit of Detection (LLD) is estimated as, the meanof the blank sample plus 2 or 3 times the SDobtained on the blank sample:

LLD = mean(blk) + Zs(blk)

where the Z-value is usually 2 or 3

Detection Limits2.Biologic Limit of Detection (BLD)= Limit

of Detection (LOD)-It is the limit of blank, after the addition of theanalyte of interest (Its lowest concentration)- BLD is estimated as the LLD plus 2 or 3 times thestandard deviation obtained from the "spiked"sample

BLD = LLD+ Zs(spk)

where the Z-value is usually 2 or 3- Results between BLD & LLD should be reportedwithout quantitation

Detection Limits3.Functional Sensitivity (FS)= Limit of

Quantification (LOQ) -The lowest concentration of target compoundsthat can be quantified confidently, that meets somepre-specified targets of imprecision, commonlyCV=20%- Several spiked concentrations must be studied todetermine the precision profile at the lowconcentration range and to select the lowestconcentration at which a 20% CV is obtained.

To Sum Up!1.LoB is the highest apparent analyte

concentration expected to be found whenreplicates of a blank sample containing noanalyte are tested.

2.LoD is the lowest analyte concentration likely tobe reliably distinguished from the LoB and atwhich detection is feasible.

3.LoQ is the lowest concentration at which theanalyte can not only be reliably detected, but atwhich some predefined goals are met. The LoQmay be equivalent to the LoD or it could be ata much higher concentration.

Reportable Range=

Analytical Measurement Range

It is the range of numeric results a method can produce without any special specimen pre-treatment, such as dilution.

Ø It should be verified, for the manufacturer’sclaim, using the linearity experiment.Ø It is performed using either calibrators,proficiency samples, or samplesØ Serial dilutions will be made covering thewhole analytical measurement range, andreaching as close as possible to the claimedvalues of the manufacturer.ØEach dilution is to be processed in duplicateto remove the element of imprecision.

Ø The observed measures (on the X-axis) areplotted against the expected measures (onthe Y-axis), and a line point to point graph isconstructed for each analyte.ØThe line is judged visually for its linearityand according to each experiment, theanalytical measurement range of eachmethod is verified, where any patient resultobtained in the future, outside the verifiedrange, cannot be released without furtherprocessing (Dilution or concentration).

Dilution Expected conc Observed conc

D1 8.27 8.27

D2 4.56 4.7

D4 2.5 2.6

D6 1.5 1.52

D7 1 1.07

D8 0.5 0.68

Correlation: r 0.999886844

Analytical range: fT4 : 0.1 - 12 ng / dl

0

1

2

3

4

5

6

7

8

9

0 2 4 6 8 10

Observed measure

Expe

cted

mea

sure

PrecisionCloseness of agreement between quantity values obtained by replicate measurements of a quantity, under specified conditions.

PrecisionØPrecision should be assessed usingquality control material (A minimum of 2levels), or pooled serum (Of minimum twoconcentration levels)ØEach level of the QC material / pooledserum is measured 5 times per day (Withinrun), for 5 days (in between runs)

PrecisionØThe measures obtained from this precisionstudy are to be collected, and the mean, SD,and CV are calculated for each parameter,for the used QC levels / pooled serum.ØThe obtained CVs, are statisticallycompared to the manufacturer’s claim usingANOVA test of significance, to determine ifthere is a significant different between theobtained CVs (and their verificationintervals), and the manufacturer’s claim at acertain CI; usually 95%

AccuracyCloseness of the agreement between theresult of a measurement and a true value ofthe measurand.

TruenessCloseness of the agreement between thereplicates of result of a measurement, anda true value of the measurand.

ØThe difference between the mean ofreplicates of a measurement, and its truevalue is the BIAS.Ø The CLSI calculation of bias is based onthe results of 7 – 11 PT samples; each ismeasured in duplicate, and then comparedto the true value (Peer’s mean) usingstudent T-test.

A concept!!!

A concept!!!

A concept!!!

Why is Z-score used in PT results??

Bias is verified for the tested method whenthere is no significant difference between:

q Mean Z-score of PT results (7 – 11) is notsignificantly different from Zero

q Mean Z-score of PT results (7 – 11) is notsignificantly different from peer’s mean

Reference intervalsØRemember that we are just “Verifying” thereference intervals stated by themanufacturer or published in the literature,and “transferring” them to the lab using themethod under study.Ø “Establishment” of reference intervals isanother issue.

Reference intervalsØAcceptability of the transfer shall beassessed by examining 20 referenceindividuals, from our subject population, andcomparing the obtained test results to thoseof the manufacturer/ Literature.ØThose 20 individuals should be selected insuch a way that will satisfy the exclusion andpartitioning criteria.ØThe test results should be examined tomake sure that none of the results appearsto be an outlier.

Reference intervals

ØThe manufacturer's / Literature referenceintervals are considered verified if no morethan two of the 20 tested subjects' values (or10% of the test results) fall outside thoseranges.

Reference intervals

ØExclusion / partitioning criteria include:age, sex, fasting status, disease history,drug history, previous surgeries, and time ofthe cycle / pregnancy for females.

Method ComparisonØ According to CLSI, at least 40 samplesshould be assayed on both methods underexamination (two field methods), or betweenone tested method and a reference method.Ø Several statistical approaches can beused, one of them is to calculate thecorrelation coefficient “r”Ø“r” should be more than or equal 0.95

Total Error

It is the summation of both Random andSystematic error.

It is calculated as follow:TE = Bias (%) + 2 CV

It is compared to Biological Variation (or anyother specifications) for Total AllowableError

UncertaintyØ It is an interval around a reportedlaboratory result that specifies the location ofthe true value with a given probabilityØ It takes into consideration both theimprecision (SD), and the inaccuracy (Bias)Ø It is calculated from the data of 6 monthminimum

What performance characteristics are usually validated?

ØReportable range (Linearity)ØPrecision (or imprecision)ØAccuracy (or inaccuracy, bias)ØReference interval