Post on 07-Feb-2017
2
Topics
Quality Assurance vs. Quality Control Quality Assurance Before, During, and After
Testing Quality Control EQA
3
Objectives
Identify requirements for QC/QA. Identify types of quality controls that must
be in place and the rationale for each. Given individual lab needs, develop a plan
for setting up an internal QC program. Identify types of quality assurance
programs and their purpose. Given individual lab needs, develop a plan
for setting up an external QA program.
6
Definitions
Term DefinitionQuality Control
The measures that must be included during each assay to verify that the test is working properly.
Quality Assurance
The overall program that ensures that the final results reported by the laboratory are accurate
Quality Assessment
A means to determine the quality of the results generated by the laboratory. It is usually an external evaluation of the laboratory's performance. Quality assessment is a challenge to the effectiveness of the QA and QC programs.
7
Quality Assurance vs. Quality Control
Quality Assurance Quality Control
Definition
Activities to ensure process are adequate for a system to achieve its objectives
Activities to evaluate a product or work result
Examples
Establish standard procedures for sample collection
Define criteria for acceptable samples
Analyze known QC sample to determine if a test is valid
Decide if a sample is acceptable for testing
8
Quality Control is Part of Quality Assurance
"The aim of quality control is simply to ensure that the results generated by the test are correct. However, quality assurance is concerned with much more: that the right test is carried out on the right specimen, and that the right result and right interpretation is delivered to the right person at the right time.”
Quality Assurance
9
QA or QC?
HIV-negative plasma control SOP writing Personnel records Phylogenetic analysis DBS condition inspection Instrument calibration Proficiency testing New operator training Assay validation Lot release testing
12
quality assurance before, during, and after testingWhat are some examples of errors that occur before, during, and after testing, and what can we do about them?
15
Preventing and Detecting Errors Before Testing
Lab personnel training and certification Sample criteria set and used Specimen labeling, collection and transport Specimen storage and processing Check storage and room temperature Lab set up Reagent inventory and expiration dates SOPs for all related procedures Preventive maintenance and equipment
checks External Quality Assessment (proficiency
testing)
17
Preventing and Detecting ErrorsDuring Testing
Conducting test according to SOPs Quality Control (QC) (in-process controls) Safety precautions Prepare reagents Correctly interpreting test results Time management and planning
19
Preventing and Detecting Errors After Testing
Re-check patient/client identifier Write legibly Clean up and dispose of contaminated
waste Complete quality control review
– Positive and negative control results– Contamination check (phylogeny)– Other QA checking (e.g. SQUAT)
Transcribing, reporting and interpreting results
Critical review of results and interpretation Record Maintenance
20
SQUAT: Sequence Quality Assessment Tool
QA tool developed for WHO by Dr. Rami Kantor and colleagues, Brown University
Open-source and freely available To be used by genotyping labs before
sending sequences to national HIVDR working groups for surveillance and monitoring activities– Labs should perform initial investigations and
take corrective actions if required Sequences can be assessed by WHO for
external quality assurance as part of ongoing accreditation process
21
SQUAT Functions
Sequence alignment HIV PR and/or RT sequence identification Mutation identification and generation of
export file for uploading to database Flagging of problematic sequences
– ambiguous bases (mixtures)– stop codons– insertions/deletions– frameshifts – genetic distance anomalies– atypical and ambiguous amino acids
23
Find It, Then Fix It
Use a problem-solving team. Investigate root causes. Develop and implement appropriate
corrective actions. Examine effectiveness. Record all actions and findings.
24
Discussion
Why do errors occur? What are some common errors that might
occur with HIVDR Genotyping? Where is QA applied in a HIVDR testing
site? What are some steps to take before,
during, and after testing to assure the quality of results?
Describe the impact that errors will have on public health.
26
What is quality control (QC)?
Monitoring the test quality itself Ensuring that the assay is working
appropriately and accurate test results are reported with confidence.
Prevention Detection Correction
27
QC: Two Levels/Two Types
Levels– Testing of samples with known results to verify
the procedure is working properly– Interpreting the presence or absence of
controls. Types
– Internal quality control – External quality control
28
Internal QC
Internal QC samples with known reactivity are included with the following HIVDR test kits.– TruGene– ViroSeq
For in-house assays, prepare and qualify your own internal QC samples
Test the IQC samples as you would test patient specimens.
Include positive and negative controls in every run.
Know what to do in case of unexpected results
29
Genotyping Assay Procedures
Patient virus
RNA Extraction
cDNA synthesis (RT)
cDNA amplification (PCR)
Electrophoresis
Data Analysis (base calling)
Report (sequence, mutations, interpretation)
Sequencing reactions
Positive and negative controls
Positive control
30
Guidelines for HIVDR Genotyping Internal QC Policies
Negative controls– HIV-negative plasma, at extraction step– Water, at RT step and at PCR step
• If any negative control shows a detectable PCR product, all samples must be discarded
– Investigate to locate source of contamination Positive controls
– HIV-positive plasma, at extraction step– HIV DNA at PCR step (low copy number)
• If positive control does not produce a PCR product, investigate to determine cause; handling of products from test samples may depend on outcome
– HIV DNA at sequencing step SOPs required
31
External QC
External QC samples do not come with the test kit and are provided by an external source – Regional reference laboratory– Commercial supplier– Another laboratory that has prepared validated
quality control in-house samples Test the EQC samples as you would test
patient specimens.
32
Internal or External Control Failure
If QCs fail, repeat the run. If problems or errors occur, then corrective
actions should be taken before results distribution.
33
Planning Your Internal QC Program
Incoming samples Incoming reagents and kits Methodology or instrument comparison
34
Incoming Samples
Upon arrival, all incoming sample shipments should be evaluated for proper collection, transport and storage
35
Incoming Reagents and Kits
Evaluate all incoming reagents and kits upon arrival.
Perform lot-to-lot comparisons.– Assay previously tested patient samples or
controls on both the previous lot and the new lot and compare the results.
– When a new lot has been evaluated and ‘passed,’ label container “Ready to Use.”
36
Lot-to-lot Comparison for Critical Reagents
Lot-to-lot comparison for critical reagents:– For commercial kits:
• Perform parallel run with new vs. existing kits• Define acceptance criteria for reagent lot to lot
comparison– For home-brew assays:
• Perform parallel run with new vs. existing lots• New lot of primers, RT-PCR enzymes, sequencing
reagents Source of specimens
– Positive controls– Previously tested samples– Old proficiency panels
37
Critical Reagent Quality Control
Extraction buffers
Enzymes (RT, DNA polymerase)
Primers
Capillary matrix polymer
Reaction buffers
RNA Purification
RT-PCR
Electrophoresis
Sequencing reactions
Results
38
Methodology or Instrument Comparison
For tests that have multiple instruments available in the laboratory for use, compare instruments.
Results should be comparable. Compare and document results routinely
(depending on lab throughput).
= ?
39
Small Group Discussion
1. What do you need to do to set up a QC program for genotyping? What components should be included?
2. If a RT-PCR negative control becomes positive, what should you do?
3. How can you perform lot-to-lot comparisons in your lab ?
4. What is critical to include in a QC SOP?5. Once you are done with a lot-to-lot
comparison and obtained satisfactory results, what should you do for the next step?
40
EQA
What is EQA and why is it important?Which EQA Methods should we employ?• Proficiency Testing• On-Site Evaluation• Re-testing
41
A Systems Approach to Laboratory Quality
Organization
Stock Managemen
t
SOP, Documents & Records
Process Improvemen
t
Personnel
Occurrence Managemen
t
Specimen Managemen
t
Equipment
Data Managemen
t
Assessment
Safety & Waste
Management
Quality Control
42
External Quality Assessment (EQA):Definition and Purpose
Objective assessment of a test site’s operations and performance by an external agency or personnel
Allows comparison of performance and results among different test labs
Provides early warning for systematic problems associated with kits or operations
Provides objective evidence of testing quality
Indicates areas that need improvement Identifies training needs
43
Management Responsibilities
Determine policies for EQA (WHO, WHAT, WHEN, HOW)
Assign responsibility Establish & maintain a system and
schedule Receive EQA results and support corrective
action measures Manage corrective action efforts Monitor and maintain records Investigate deficiencies Communicate outcomes
44
Testing Personnel’s Responsibilities: Overview
Participate in the EQA program Take corrective actions Maintain EQA records Communicate outcomes to supervisors
46
What is Proficiency Testing?
Panels of specimens are sent to multiple test labs by reference laboratory/EQA provider
Test labs perform tests and report results
Results indicate quality of personnel performance and test lab operations
Results are often compared across several testing labs
47
Proficiency Panels for HIVDR EQA
VQA (NIH-supported, Rush University, Chicago1)
TAQAS (TREAT Asia2) ENVA/QCMD3
ANRS France DigitalPT/Accutest Acrometrix Others?
1. Huang et al. J Clin Microbiol.43(8):3963-70 Aug 20052. Land et al. Journal of Virological Methods 159(2): 185-193; Aug 20093. Pandit et al. J Clin Virol. 43(4): 401-6; Dec 2008
48
What is On-site Evaluation?
Periodic site visits to perform systematic assessment of lab practices– Focuses on how the lab
monitors its operations and ensures testing quality
– Provides information for internal process improvement
49
What is On-site Evaluation? – Cont’d
Also referred to as audits, assessments, or supervisory visits
Learn “where we are” Part of every lab quality
system Measures gaps or
deficiency Collect information for:
– Planning & implementation– Monitoring– Continuous improvement
50
What is Re-testing?
The process by which a random selection of specimens are collected from the routine workload at the test lab and sent to the reference laboratory for validation
Used to detect errors
53
Small Group Discussion
1. What are the methods in a EQA program?2. What type of EQA is most appropriate in
our own situation?3. What is the potential of internal bias in
self-evaluation of quality?
57
Tester Responsibilities: Ensuring a Productive Site Visit
Before Visit:– Record keeping is essential. Get organized– Confirm date of visit– Review written policies and procedures– Conduct internal assessment in preparation of
site visit During Visit
– Participate in visits – cooperate– Ask questions of site assessors
After Visit– Take corrective actions, where necessary
Remember – visits are instructive, not punitive
58
On-site Evaluation: Pre-Evaluation Preparation
Assign Responsibility– Laboratory management – Quality Manager
Use Integrated team approach Determine who will conduct on-site
evaluations– Select auditors with necessary skills:
• Attention to detail• Ability to communicate effectively• Diplomacy
– Provide appropriate training Schedule site visits
59
On-site Evaluation: Pre-Evaluation Preparation
Determine site visit frequency:– Established sites - twice per year/once per year
For efficiency, cluster evaluations geographically
Schedule in advance– Announced or unannounced visits
60
On-site Evaluation: Entrance Interview
The entrance interview sets the tone for the entire visit
Provide overview of process in terms of what will be done– Review of facility– Record review– Observation– Interview with testing staff– Use of proficiency panel– Exit interview
61
On-site Evaluation: Information Gathering
Observe physical layout of the lab Evaluate testing operations
– Specimen collection– Observation of test performance– Quality control– Inventory of kits– Record-keeping
Use quality systems checklist Conduct in an instructional, not punitive,
manner.
62
On-site Evaluation: Outcome Assessment
Evidence ofimplementation
& maintenance of quality system?
• Ensure visit is sufficient to objectively document evidence
• Expand visit or observation
• Reference deficiencies
• Conclude visit
• No deficiencies
YES NO
63
On-Site Evaluation: Exit Conference
Review findings with supervisory and testing staff
Make positive statements first – acknowledge staff cooperation and support
Address negative findings – allow test site to discuss findings and provide additional information
Provide instructions and timeframe for submitting plan for correcting problems
64
On-site Evaluation: Reporting
Include information:– Lab Name & Location– Date of Visit– Assessment Team Members– Major Findings– Recommendations for corrective actions
Submit completed checklist and report to relevant authorities
65
Minimal Package for QC/QA
WHO-accredited DR labs should be enrolled in at least one EQA program
Perform self-assessment at least once a year
Internal/external controls must be included in each assay run and the results recorded.– Trending over time can be a very useful quality
indicator Clearly define valid testing run criteria in
related SOPs
66
Discussion
What is EQA and why is it important? Which EQA Methods should we employ?
– Proficiency Testing– On-Site Evaluation– Re-testing
67
Reflection
What are our responsibilities in preventing, detecting, and correcting errors before, during, and after testing?
68
Summary
Identify requirements for QC/QA. Identify types of quality controls that must
be in place and the rationale for each. Given individual lab needs, develop a plan
for setting up an internal QC program. Identify types of quality assurance
programs and their purpose. Given individual lab needs, develop a plan
for setting up an external QA program.