IQC Edward Kearney. Why is IQC/EQA important? To ensure the test result is fit for purpose.
-
Upload
stephany-russell -
Category
Documents
-
view
235 -
download
2
Transcript of IQC Edward Kearney. Why is IQC/EQA important? To ensure the test result is fit for purpose.
IQC
Edward Kearney
Why is IQC/EQA important?
To ensure the test result is fit for purpose
EQAHistoric
Accuracy
Repeated samples may give precision
Who determines if a patient gets a statin?
We do !
Who determines if a patient gets a Dx of DM?
We do !
Who determines if a patient gets a prostrate biopsy?
We do !
Who determines if a patient gets dialysis?
We do !
How do we know we are right?
Do we?We do !
Westgard .com “1575”
Diabetes
Fasting Plasma Glucose > 7.0 mmol/L
On two or more occasions can make the diagnosis of Diabetes Mellitus
Fasting Glucose 6.8 mmol/L = IFG
Fasting Glucose 7.2 mmol/L = DM
Acute Coronary SyndromeTroponin T
> 0.05 ug/L = Biochemical evidence of cardiac damage
< 0.05 ug/L = No Biochemical evidence of cardiac damage
How do we know we are right?Standardisation
Calibrators
Test conditions
Reagents
Quality Control
Internal Quality Control (IQC)
External Quality Assurance (EQA)
IQC
Statistical QC
Electronic QC
Real time
Stable process
Bias
Precision
IQC Materials
Mimic Patients sample
Cover normal and pathological range
Ideal – decision points
Controlling the quality - IQC
Compare process performance with what is expected under stable operation
Stable performance determined by assaying control material over time : Calculate mean and SD
Measurements are made continually and compared with the original distribution
Unexpected results are identified to alert the analyst to possible changes in process performance
Multi-rule QC
Westgard rules
A multi-rule Shewhart chart for quality control in clinical chemistry –
JO Westgard, PL Barry, MR Hunt, T Groth - Clinical Chemistry, 1981
Use and Interpretation of Common Statistical Tests in Method-Comparison Studies - all 2 versions »
JO Westgard, MR Hunt - Clinical Chemistry, 1973
Performance characteristics of rules for internal quality control: probabilities for false rejection
JO Westgard, T Groth, T Aronsson, H Falk, CH de … - Clinical Chemistry, 1977
Assuring analytical quality through process planning and quality control.
JO Westgard - Arch Pathol Lab Med, 1992
European specifications for imprecision and inaccuracy compared with operating specifications
JO Westgard, JJ Seehafer, PL Barry - Clinical Chemistry, 1994
Internal quality control: planning and implementation strategies
JO Westgard - Annals of Clinical Biochemistry, 2003
The Quality of Laboratory Testing Today, An Assessment of σ Metrics for Analytic Quality Using Performance Data From Proficiency Testing Surveys and the CLIA Criteria for Acceptable Performance
James O. Westgard, and Sten A. Westgard. Am J Clin Pathol 2006;125:343-354
• Determine the expected distribution of control values• Calculate mean and SD from control data to establish
control limits for control chart• Expect control values to fall with certain control limits
– 95% within 2 SD– 99.7% within 3 SD
• Plot control values versus time to provide control chart• Identify unexpected values
Very Unexpected
Somewhat Unexpected
XX
X X XX
XX
XX X
X
XXX
XX
XXX
XXXX
250 260245240 255235 265
265
250
260
255
245
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 20 1 2 3 4
Run Number (or Time, Date)
240
235
X
XX
X
X X
X
X
X
X
XX
X
X
X X
XX
X
X
X
X
X
X
Conceptual basis of the control chart
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
QC Data
12s
13s 22s R4s 41s 10x
Report Results
Take Corrective Action
Flowchart and logic for the multirule IQC procedure commonly known as “Westgard Rules”
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
IQC2 SD rule for 1 QC sample
Statistically 1 in 20 will be outside this range
This is not unexpected
5% of runs (false) rejected
IQC
2 SD rule for 3 QC samples
This is not unexpected
18% of runs (false) rejected
What actually happens?
• EM Kearney. The South Thames (East) Quality Assurance Liasion Group (1998)
• James O Westgard. Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
• David Housley. North Thames audit and QA group: Audit of Internal Quality Control practice and processes. (2005)
5. NUMBER OF SAMPLES USED TO ASSIGN QC VALUES
• 1 = provisionally 3, then re-assessed at 15• 2 = 10 – 15• 1 = 12• 11 = 20 (one 20 days ?); one with minimum of 1
month run in (major variations discussed with manufacturer); one re-assess after 1000; some variation in number of batches / days etc
• 1 = 20 - 25• 1 = > 30• 3 = at least 50• 1 = adjusted every month 9 ( no number stated)
David Housley. North Thames audit and QA group: Audit of Internal Quality Control practice and processes. (2005)
EP5-AClinical and Laboratory Standards Institute (NCCLS) EP5-AEvaluation of precision performance of Clinical Chemistry Devices: Approved guidelines.Precision Evaluation ExperimentRecommendationsMinimum of 20 working days
Batch analysers: two runs per day with two test samples at each of at least two levels.Random access: four samples at each level should be analysed through out the day.
At the end of each 5 days the control limits are recalculated and all data checked for acceptability.The cause of outliers should be determined.Data may not be rejected without valid justification.Learning curve data may be rejected and replaced by a equal number of points at the end.Other methods:Not recommended:
Single run of 20 samples for within runSingle observation each day for 10 – 20 days for day to day
Because:Single runs do not reflect usual operating parameters, thus adversely affecting the estimate.Single observations will be highly dependent on the number of days used.
EP5-ARandom access:
Four samples at each level should be analysed through out the day.
At the end of each 5 days the control limits are recalculated and all data checked for acceptability.
The cause of outliers should be determined.
Data may not be rejected without valid justification.
Learning curve data may be rejected and replaced by a equal number of points at the end.
Criteria use to determine acceptability
• Written procedure 1
• Professional judgement 6
• Combination of above 12
EM Kearney the South Thames (East) Quality Assurance Liasion Group (1998)
Is the procedure always adhered to?
• Yes 2
• Mostly 1
• No 5
• Blank 11
EM Kearney the South Thames (East) Quality Assurance Liasion Group (1998)
QC rules used for calcium?2SD 6
3SD 1
12s + Multi-rules 1
Westgard 3
Limits professional judgement, Vitros F2/F3
EM Kearney the South Thames (East) Quality Assurance Liasion Group (1998)
Calcium QC rule violated describe the process for dealing with this
Check QC 3
Rerun 12
Recalibrate 4
Inspect QC history 1
EM Kearney the South Thames (East) Quality Assurance Liasion Group (1998)
Westgard Rules
1. Check control 2 Make up new
1. Repeat 2 Make up new
1. Repeat 2 Recalibrate
EM Kearney the South Thames (East) Quality Assurance Liasion Group (1998)
9. ARE WESTGARD RULES USED AS PRIMARY FORM OF DATA EVALUATION?
• YES = 18 (62%)• NO = 11 (38%)
– 1 = Mean and target range (< 2SD)– 5 = single 2SD rule– 1 = single 1.5 SD rule– 1 = not formally but simplified Westgard used– 1 = under consideration– 1 = Instrument set flag points, but no rule stated– 1 = 2SD in first instance, then Westgard for detail bias
profile
David Housley. North Thames audit and QA group: Audit of Internal Quality Control practice and processes. (2005)
REASONS FOR ACCEPTING FAILED QC
• Only if not clinically significant – 7 sites• Lack of reagent prevents change• Lack of staff, pressure to reduce TAT, demotivated staff• Checked with EQA• Various. eg. very close to limit, one QC in/ one QC out, QC
deterioration before new QC is ready e.g. lot no. change until new range established.
• Only if all attempts to correct it have failed• Only after consultation with manufacturer• Only if calibrant returns result to normal• Only if no reason can be found and 1 QC out• eg. If LDH QC is out, the reagent is changed the next day
David Housley. North Thames audit and QA group: Audit of Internal Quality Control practice and processes. (2005)
Planning strategies (doing the right IQC)
Define the quality requirement for the test
Determine the method precision and bias
Identify candidate IQC procedures
Predict IQC performance
Select goals for IQC performance
Select an appropriate IQC procedure
Clinical Outcome Criteria (DInt)
Analytical Outcome Criteria (TEa)
Operating Specifications(smeas, biasmeas, control rules, N)
ProficiencyTestingCriteria
MedicallyImportantChanges
TotalBiologic
Goals
IndividualBiologic
Goals
Performance Criteria SDMax, BiasMax
DiagnosticClassification
Goals
Stateof the
Art
ArbitraryControl
A system of quality requirements and operating specifications
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
12s
12.5s
13s/22s/R4s
13s
13.5s
N
2
2
2
2
2
R
1
1
1
1
1
Thur 11/15/1 - 09:44:08
Systematic Error (SE, multiples of s)
Pro
bab
ilit
y fo
r R
ejec
tion
(P
)
Power Function Graph (SE)
Pfr given byy-intercept
Ped depends on size of
systematic error
Power curves for commonly used IQC procedures having two control measurements per run. Pfr, or the probability for false rejection, is given by the y-intercept. Ped, the probability for error detection, depends on the size error occurring, which is illustrated by the vertical line, and estimated by reading the y-value at the point of intersection with the power curve.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
13s/2of32s/R4s/31s/6x
13s/22s/R4s/41s
12.5s
12.5s
13s/22s/R4s
13s
13.5s
13s
N
6
4
4
2
2
2
2
1
R
1
1
1
1
1
1
1
1
Mon 4/9/1 - 13:18:00
Pro
bab
ilit
y fo
r R
ejec
tion
(P
)
Power Function Graph (SE)
Systematic Error (SE, multiples of s)
Probabilityof false
rejection, Pfr given by
y-interceptsProbability of
error detection, Ped , depends
on size of systematic error
Pfr is0.07to
0.01
Ped is 1.00 to 0.40
Example use of a power function graph to assess IQC performance for a cholesterol test where the medically important systematic error is equivalent to 2.85 times the standard deviation of the method.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
13s/2of32s/R4s/31s/6x
13s/22s/R4s/41s
12.5s
12.5s
13s/22s/R4s
13s
13.5s
13s
N
6
4
4
2
2
2
2
1
R
1
1
1
1
1
1
1
1
Mon 4/9/1 - 13:18:00
Pro
bab
ilit
y fo
r R
ejec
tion
(P
)
Power Function Graph (SE)
Systematic Error (SE, multiples of s)
Lines show SE detectable with
90% chanceby different
QC procedures
Sizes of systematic errors that can be detected with 90% assurance by different IQC procedures
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
0.0 1.0 2.0 3.0 4.0 5.0
13s/2of32s/R4s/31s/6x
13s/22s/R4s/41s
12.5s
12.5s
13s/22s/R4s
13s
13.5s
13s
N
6
4
4
2
2
2
2
1
R
1
1
1
1
1
1
1
1
Mon 1/6/3 - 14:09:43
Pfr
0.07
0.03
0.04
0.03
0.01
0.00
0.00
0.00Operating Point
All
owab
le I
nac
cura
cy (
bia
s, %
)
Allowable Imprecision (s, %)
Mathematical basis for a chart of operating specifications that shows the bias and imprecision that are allowable for different IQC procedures, whose control rules and Ns are given in the key at the right.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
0.0 1.0 2.0 3.0 4.0 5.0
13s/2of32s/R4s/31s/6x
13s/22s/R4s/41s
12.5s
N
6
4
4
R
1
1
1
Mon 1/6/3 - 14:19:03
Pfr
0.07
0.03
0.04Operating Point
All
owab
le I
nac
cura
cy (
bia
s, %
)
Allowable Imprecision (s, %)
OPSpecs Chart TEa 10% with 90%AQA(SE)
Plo
t your o
bserv
ed C
V and b
ias
Application of the OPSpecs tool by plotting the observed method bias as the y-coordinate and the observed method precision as the x-coordinant to describe the “operating point” of an analytical method.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
1. Define quality requiredfor diagnostic test, %TEa
2. Assess method%bias, %CV
3. Calculate normalizedoperating point
4. Plot on normalizedOPSpecs charts or EZ rules3
5. Inspect normalized OPSpecs charts
6. Select control rules,number of measurements
7. Adopt Total QC strategy
8. Reassess for changes
A practical procedure for planning and selecting IQC procedures.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
HI-PedStrategy
IQC
Other QC
QI
MOD-PedStrategy
IQC
Other QC
QI
LO-PedStrategy
IQC
Other QC
QI
Total quality control strategies (TQC) that illustrate the proper combination of IQC with instrument and function checks (Other QC) and need for quality improvement (QI).
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
Minimize cost ofstatistical QC
Minimize cost ofnon-statistical QC
Maximizeerror detection
Maximizenon-statistical QC
Improve methodperformance
Document TQCsystem
Maximizeerror detection
Maximizenon-statistical QC
Improve methodperformance
Optimize QC forprocess stability
Deploy skilledanalysts
Add patientdata QC
Apply QP processwith OPSpecs
charts
AQA orPed
HI-Ped (90%)
MOD-Ped (50%)
LO-Ped (<50%)
A detailed flowchart to guide the development of TQC strategies.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
12.5s
13s
13.5s
14s
N
2
2
2
2
R
1
1
1
1
Thur 8/30/1 - 09:49:08
Pro
bab
ilit
y fo
r R
ejec
tion
(P
)
Critical-Error Graph (SE)
Systematic Error (SE, multiples of s)
Albumin
CO2
Chloride
OTHER TESTSCrit SE > 4.0
Creatinine, T Bil, UAPotassium, GGT,
ALP, PhosphorousBUN, CholesterolAST, LD, Glucose
Total Protein
Calcium
Results of an example application of IQC design for a multitest chemistry analyzer.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
Select QC design
from drop-down
list
Select material
from drop-down
list
Entercontrolresult
Programcalculates
z-value
Out-of-controlidentified by
red bar
STARTUPQC design uses
multirule
MONITORQC design uses
2.5s limits
Programdisplaysz-valueon scale-4 to +4
Multi-chart displayshows multiple materials
and multiple designs with multi-rule or single-rule
QC procedures
Implementation of a multistage IQC strategy.
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–611
Interfaces orManual data entry
Laboratory testing
processes
Internal QCAccept/Reject
EQA data transfer
EQAData Analysis
IQC PlanningAnd Design
Inservice training
Validation tools
DocumentationValidation Reports, Peer Reports
Toubleshooting
Inventory, ordering
Production, shippingSupplies & materials
Internet SupportServices
A Total Quality Control Support System that integrate IQC and EQA
James O Westgard Internal quality control: planning and implementation strategies Ann Clin Biochem 2003; 40: 593–
611
What to do?• Do define the quality required for the test
• Do select QC procedures that minimise false rejections
• Do select QC procedures that detect medically important errors
• Do adopt modern QC planning tools and techniques
• Do standardise QC operations
• Do calculate control limits from your own laboratory data
• Do provide computer support to analyse and interpret QC data
• Do reject out-of-control runs, identify the problem, and eliminate the cause
• Do adopt a Total QC strategy to maximise the cost-effectiveness of QC
• Do calculate daily/weekly/monthly patient means
What not to do?
• Don’t use 2SD control limits
• Don’t just repeat the controls
• Don’t use the same control rules for all tests
• Don’t use bottle values to calculate control limits
• Don’t use medical decision limits as control limits
• Don’t rely on electronic QC alone
• Don’t eliminate SQC in POC applications
Additional methods of IQC
Average of Normals (AoN, Daily patient means)
Standard signals
Experience
Expertise
IQMMonitors system performance real time
Automatically detects potential failures that affect the analytical performance
Automatically performs corrective action
Automatically documents failures and actions taken
Provides quality control reports
IL GEM Premier 4000