Risk Assessment Tools for Identifying Hazards and
Evaluating Risks Associated with
IVD AssaysRobert C. Menson, PhD
AACC Annual MeetingPhiladelphia, PA
22 July 2003
© Menson & Associates, Inc AACC 2003 Edutrak 2
What Risks Must Be Managed?
Risk to safety of patients, users, handlers
Business Regulatory
Product liability
© Menson & Associates, Inc AACC 2003 Edutrak 3
Elements of the Risk Management Process
Risk Analysis
Risk Evaluation
Risk Control
Post-Production Information
Ass
essm
ent
Man
agem
ent
© Menson & Associates, Inc AACC 2003 Edutrak 4
Risk Assessment Tools
Risk MatrixFTA=Fault Tree AnalysisPHA= Preliminary Hazard AnalysisFME(C)A=Failure Mode Effects (Criticality) AnalysisHAZOP=Hazard Operability AnalysisHACCP=Hazard Analysis and Critical Control Point
© Menson & Associates, Inc AACC 2003 Edutrak 5
Applications of Risk Analysis
DesignProductionPremarket NotificationsComplaintsMDRChange ControlFailure AnalysisEtc.
© Menson & Associates, Inc AACC 2003 Edutrak 6
Risk ManagementLife Cycle
Concept & Feasibility Development Scale-Up & Transfer Production
System Level Assessment
Design Assessment
Process Assessment
Customer FeedbackAssessment
© Menson & Associates, Inc AACC 2003 Edutrak 7
Risk Management& Design Control
DesignControl
RiskAssessment
Requirements Plan SpecificationsTest
Methods &Results
ProductionsMethods
Change Records
RiskManagement
Plan
PreliminaryHazard
Analysis
Detailed Analysis(FMEA, FTA, HACCP, HAZOP)
RiskManagement
Report
Risk Reviews
DevelopmentPlanningConcept& Feasibility Scale-Up & Transfer Production
© Menson & Associates, Inc AACC 2003 Edutrak 8
Key Concepts of Risk
The frequency of the potential harm;How often the loss may occur;
The consequences of that loss;How large the loss might be;
The perception of the loss;How seriously the stakeholders view the risk that might affect them.
© Menson & Associates, Inc AACC 2003 Edutrak 9
Risk Management Terms
Intended Use/PurposeHarmHazardRiskResidual RiskTolerable Risk
© Menson & Associates, Inc AACC 2003 Edutrak 10
Intended Use/Purpose
Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer
ANSI/AAMI/ISO 14971:2000, definition 2.5
© Menson & Associates, Inc AACC 2003 Edutrak 11
Harm
Physical injury or damage to health of people, or damage to property or the environment.
ISO/IEC Guide 51:1999, definition 3.3“Guidelines for inclusion of safety aspects in
standards.”
© Menson & Associates, Inc AACC 2003 Edutrak 12
Hazard
Potential source of Harm
ISO/IEC Guide 51:1999, definition 3.5
© Menson & Associates, Inc AACC 2003 Edutrak 13
Risk
Combination of the probability of occurrence of harm and the severity of harm.
ISO/IEC Guide 51:1999, definition 3.2
© Menson & Associates, Inc AACC 2003 Edutrak 14
Residual Risk
Risk remaining after protective measures have been taken.
ISO/IEC Guide 51:1999, definition 3.9
© Menson & Associates, Inc AACC 2003 Edutrak 15
Tolerable Risk
Risk which is accepted in a given context based on the current values of society.
ISO/IEC Guide 51:1999, definition 3.7
© Menson & Associates, Inc AACC 2003 Edutrak 16
Hazard Identification
HazardIdentifcation
BrainstormingPHAFTA
FMEA
LawsCodes
Standards
EventsMDRs
AccidentsEtc
Listof
Hazards
© Menson & Associates, Inc AACC 2003 Edutrak 17
Risk Matrix
HIITOccasional
IILTRemote
HHILProbable
HHILFrequent
SevereMajorMinorNegligible
SeverityFrequency
© Menson & Associates, Inc AACC 2003 Edutrak 18
ALARP CurveIn
crea
sing
Pro
babi
lity
of O
ccur
renc
e
Intolerable Region
ALARP
BroadlyAcceptable
Region
MaximumTolerable
Risk
Increasing Severity of Harm
© Menson & Associates, Inc AACC 2003 Edutrak 19
FMEA vs FTAFMEA FTA
Assumes failure of the functionality of a productIdentifies part/module failure as cause of functional failure
Assumes component or part failureIdentifies functional failure as a result of part failure
© Menson & Associates, Inc AACC 2003 Edutrak 20
FMEA vs. FTADone for entire design
Systematic way to predict new problems
A bottoms-up analysis
People expect the same results from FTA which is not true
Too difficult to do for entire designSystematic way to predict causes for usually know problems
A top down analysis
People do not expect the same results from and FMEAOften a fault tree is used for a problem or an accident.
FMEA FTA
© Menson & Associates, Inc AACC 2003 Edutrak 21
FTAAssumes fault and analyzes possible causesConnection tool for PHA* to subsystems or modulesTop downDeductiveEvaluate system (or subsystem) failuresConsidered more structured than FMEAGraphical presentation--visual picture*Preliminary Hazard Analysis
© Menson & Associates, Inc AACC 2003 Edutrak 22
FailuresPrimary--Due to internal causes that include poor design or use of inappropriate materialsSecondary--Due to failures in the operation that include equipment failureControl--Due to failures in the systems that are in place to protect the quality and safety
e.g. raw material outside specificationfailure of safety switchfailure of test method
© Menson & Associates, Inc AACC 2003 Edutrak 23
FTA Basic SymbolsBasic Flow
FAULT
AND
OR
Fault in a box indicates that it is a result of subsequent faults
Connects a preceding fault with a subsequent fault that could cause a failure
Connects two or more faults that must occur simultaneously to cause the preceding fault
© Menson & Associates, Inc AACC 2003 Edutrak 24
FTA During Design P O O R F ITIN H A N D
D O E S N O TR E S T R IC TS P IL L A G E
O R
O R
A S P E C T R A T IOT O O H IG H
M A J O RD IA M E T E R
T O O L A R G E
A S P E C T R A T IOT O S M A L L
B A S EF L A T N E S S
B A S E S IZ ET O O S M A L L
R IM T O F IL LIN S U F F C IE N T
CompetingDesign Criteria
© Menson & Associates, Inc AACC 2003 Edutrak 25
FTA Lab Failure
Patient Death OR
Other
Lab Error OR
Outliers
Systematic
Random
OR
CalibrationError
Interference
Other
© Menson & Associates, Inc AACC 2003 Edutrak 26
FTA During ReliabilityHAZARD
SYSTEMFAILURE
DRIFT>LIMIT
CMPT A FAILS CMPT B FAILS CMPT C DRIFTS REFERENCEDRIFTS
AN
D
OR
OR 4. x 10-9
4. x 10-91. x 10-16
3. x 10-9 1. x 10 -92. X 10 -85. x 10 -9
© Menson & Associates, Inc AACC 2003 Edutrak 27
What is FMEA?What is FMEA?
FMEAFailure Mode and Effects Analysis
FMECAFailure Mode Effects and Criticality Analysis
FMEAFailure Mode and Effects Analysis
FMECAFailure Mode Effects and Criticality Analysis
© Menson & Associates, Inc AACC 2003 Edutrak 28
What is FMEA?
Powerful prioritization toolInductiveHigh effective tool for identifying critical quality attributesHigh structuredMethodicalBreaks large complex designs into manageable steps
© Menson & Associates, Inc AACC 2003 Edutrak 29
FMEA
Bottom up approachEvaluates specific failuresDetailed analysis tool
Use in conjunction with PHA and FTA Complements FTA
May lead to different failure results
© Menson & Associates, Inc AACC 2003 Edutrak 30
Advantages of FMEA
Less analyst dependent than FTAAllows direct criticality assessment of componentsValuable troubleshooting aidIdentifies areas of weak designIdentifies areas of high riskPrevention planningIdentifies change requirements
© Menson & Associates, Inc AACC 2003 Edutrak 31
Disadvantages of FMEADoes not consider operator errorTediousMay not apply to all systems--especially softwareMay require extensive testing to gain informationMay miss some failure modes
Time pressuresInformation missing
© Menson & Associates, Inc AACC 2003 Edutrak 32
DefinitionsCriticality -- Weighting of hazard severity with the probability of failureSeverity -- Seriousness of effect through its impact of the system functionOccurrence -- Likelihood a specific failure will be caused by a specific cause under current controlsVerification -- Ability of the current evaluation technique to detect potential failure during designDetection -- Ability of the current manufacturing controls to detect potential failure before shipping
© Menson & Associates, Inc AACC 2003 Edutrak 33
DefinitionsDefinitions
Risk Priority Number (RPN)
= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)
Risk Priority Number (RPN)
= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)
© Menson & Associates, Inc AACC 2003 Edutrak 34
FMEA ExampleP O T E N T IA L F A IL U R E M O D E A N D E F F E C T S A N A L Y S IS
Im u n n o a s s a y D e v ic e F M E A N u m b e r S u b s y s te m P a g e o f X C o m p o n e n t _ S u p p o rt_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ R e s p o n s ib ility P re p a re d B y P ro c e s s F M E A D a te (O rig .) (re v .) C o re T e a m : _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
D e vic e / P o te n tia l P o te n tia l S P o te n tia l O C u rre n t D R R e c o m m e n d e d R e s p o n s ib ilit
y A c tio n R e s u lts
F u n c tio n F a ilu re E ffe c t(s ) C a u s e (s ) C o n tro ls P A c tio n (s ) a n d T a rg e t A c tio n s R M o d e o f F a ilu re o f F a ilu re N C o m p le te D a te T a k e n S O D P
N S u p p o rt S y s te m
H o ld s A n tib o d y
-
S e p a ra te s A n a ly te
F a ils to S e p a ra te a n a ly te
N o A n s w e r E rro n e o u s R e s u lt F a ls e H i F a ls e L o
4
79
W ro n g M a te ria l
3
33
5
55
6 0
1 0 5 1 3 5
D e v e lo p s C o lo r
© Menson & Associates, Inc AACC 2003 Edutrak 35
Process FMEA Process FMEA Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for
reducing occurrence or increasing detection of the failure conditions
Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for
reducing occurrence or increasing detection of the failure conditions
© Menson & Associates, Inc AACC 2003 Edutrak 36
Sources of Process Defects?
Sources of Process Defects?
Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts
Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts
Adjustment errorAdjustment errorProcessing wrong Processing wrong work piecework pieceMisMis--operationoperationEquipment not set Equipment not set up properlyup properlyTools and fixtures Tools and fixtures improperly preparedimproperly prepared
© Menson & Associates, Inc AACC 2003 Edutrak 37
FMEA Summary
Powerful tool for summarizing:Important modes of failureFactors causing these failuresEffects of these failuresRisk prioritizationIdentifying plan to control and monitorCataloging risk reduction activities
© Menson & Associates, Inc AACC 2003 Edutrak 38
HAZOP
Haz ard and Op erability StudyBottom up analysisDeviations from design intentionsSystematic brainstorming based on guide words
© Menson & Associates, Inc AACC 2003 Edutrak 39
HAZOPGuide Words
No/NotMoreLessAs well asOther than
© Menson & Associates, Inc AACC 2003 Edutrak 40
Hazop ModelDesign Statement
MaterialActivity Destination
Transfer Powder Hopper
© Menson & Associates, Inc AACC 2003 Edutrak 41
HAZOP
LiquidWrong powder
Other than
Larger tankInaccurate gage
Pump fastMore
Valve closedHopper full
Tank emptyValve closedLine blockedPump broken
NoHopperPowderTransfer
© Menson & Associates, Inc AACC 2003 Edutrak 42
HAZOP Plan
Interlock
Operator trainingPM
Low
Med
Med
Valve closedLine blockedPump broken
Powder flow
NO
WhoActionRiskCausesDeviationGuide
© Menson & Associates, Inc AACC 2003 Edutrak 43
HACCP
Risk Management SystemBiological HazardsChemical HazardsPhysical Hazards
RequiresPrerequisite Quality System Program Traditionally GMPs
© Menson & Associates, Inc AACC 2003 Edutrak 44
HACCP Steps1. Conduct hazard analysis and identify
preventive measures2. Identify Critical Control Points.3. Establish critical limits4. Monitor each critical control point5. Establish corrective action to be taken when
deviation occurs6. Establish verification procedures7. Establish record-keeping system
© Menson & Associates, Inc AACC 2003 Edutrak 45
HACCP Decision TreeP re v e n tiv eM e a s u re s
E x is t
S te p E lim in a teo r R e d u c e
C ritic a lC o n tro l P o in t
Is c o n tro lN e c e ss a ry fo r
S a fe ty
S to pN o t a C C P
M o d ify S te p
N o
N o
Y e sY e s
Y e s
E x ce s sC o n ta m in a tio n
P o s s ib le
N o
N o
S u b s eq u en tS te p s e lim in a te
o r R e d u c eY e s
Y e s
N o
© Menson & Associates, Inc AACC 2003 Edutrak 46
HACCP Worksheet
Biological
Chemical
Physical
Is this step a critical control point? (Y/N)
What preventative measures can be applied to prevent
the significant hazards?
Justify your
decisions for column
3.
Are any potential safety
hazards significant?
(Y/N)
Identify potential hazards
introduced, controlled or
enhanced at this step(1)
Material/processin
g step
654321
Product Description:
Method of Storage and Distribution
Intended Use and Consumer
Firm Name:
Firm Address:
© Menson & Associates, Inc AACC 2003 Edutrak 47
HACCP Plan
(7) Who
(6)Frequency
(5)How
(4)What
(10)Verificat-
ion
(9)Record
s
(8)Correct-
iveActions
Monitoring
(3)Critical
Limits for each
Action
(2)Significant Hazards
(1)Critical Control Point
Product Description:
Method of Storage and Distribution:
Intended use and Consumer:
Firm Name:
Firm Address
© Menson & Associates, Inc AACC 2003 Edutrak 48
Contact Information
Robert C. Menson, Phd
Menson & Associates, IncQuality Systems and Compliance
For FDA Regulated IndustriesPO Box 480
Newport, RI 02840Phone: 1.401.847.6320
Email: [email protected]
© Menson & Associates, Inc AACC 2003 Edutrak 49
IVD Hazards, Patient Harmand Risk Controls
Donald M. Powers, PhD
AACC Annual MeetingPhiladelphia, PA
22 July 2003
Top Related