MEDTECH 2013 Closing Plenary, Andy Shaudt, Director of Usability Services, National Center for Human...
-
Upload
medtechassociation -
Category
Technology
-
view
962 -
download
0
description
Transcript of MEDTECH 2013 Closing Plenary, Andy Shaudt, Director of Usability Services, National Center for Human...
Design and Development of Medical Devices through a Human Factors and
Usability Lens
Andy Schaudt, M.S., M.B.A.Director of Usability ServicesNational Center for Human Factors in HealthcareMedStar Institute for Innovation
October 8, 2013
How on earth do we design technologies/systems around drivers?
Do we give them bigger mirrors for putting on makeup?
Do we provide them a better steering wheel so they can drive with their knee?
Drivers are already at a disadvantage because they are human, let’s not make it more difficult/unsafe for them…
Human Error in Healthcare
“More than 1 million patients are injured and approximately 180,000 die each year” due to medical error – roughly equivalent to three commercial airliners crashed every two days. (Andrews LB, Stocking C, Krizek T, et al. 1997)
8th leading cause of death in the United States (U.S.), ahead of motor vehicle accidents, breast cancer and AIDS. (Leape 1994)
About half of adverse events were judged preventable with ordinary standards of care. (Vincent, Neale, Woloshynowych, 2001)
$17.1 Billion annual cost (Van Den Bos, et al. 2011)
Might be 10x more than previous research has shown (Claussen, et al. 2011)
Human Resources?
Engineering People’s Brains?
What is Human Factors Engineering?
“The scientific discipline concerning understanding of interactions among humans and other elements of a system, and applying theory, principles, data, and other methods to design in order to optimize human well-being and overall system performance.” As defined by the Human Factors and Ergonomics Society.
Human Factors
Psychology
Industrial Engineering
Cognitive Science
Anthropometry Biomechanics
Human Factors and Usability Benefits in Healthcare
• Can save lives• Increase safety by reducing human error• Improve system/device design• Improve reputation• Increases user acceptance• Facilitates a stronger regulatory submission• Believe it or not, it reduces overall costs…
Systems Approach
• Truly integrated systems result in higher performance, higher reliability, improved safety– Humans are one component in a larger system– Focus on the interaction or interface between people and
the equipment and environment– Fit the tools and environment to the person; not the
person to the tools and environment
Intended Users?Healthcare Providers? Patients and Family Members?
Intended Environments?
Shift in Healthcare
• Expansion in ambulatory care• Technology allowing for in-home care/monitoring• Conversion in legacy systems to more mobile-based
technology• Bottom line – users and environments are expanding,
and users’ roles are changing• Incorporating human factors and usability early and
throughout device design is crucial
Primary Problem
• IOM Report in 2000– Govt – 50% less errors in 5 years
• 13 years later…– Essentially NO CHANGE
• Why?– Focus still on individual performance– Solutions inconsistent with safety
science
Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. May 18 2005;293(19)Wachter RM. The end of the beginning: Patient Safety Five Years After 'To Err Is Human'. Health Aff. 2004(11)Wachter RM. Patient Safety At Ten: Unmistakable Progress, Troubling Gaps. Health Aff. 2010 (29:1)Landrigan, Parry, et al. Temporal Trends in Rates of Patient Harm Resulting from Medical Care. NEJM 363(22): 2010 Shekelle, Pronovost, et al. Advancing the science of patient safety. Ann Int Med 154(10): 2011Longo, Hewett, Ge, Schubert. The long road to patient safety: a status report on patient safety systems. JAMA, 294(22): 2005.
Error• Human error is:
“A generic term to encompass all those occasions in which a planned sequence of mental or physical activities fails to achieve its intended outcome, and when these failures cannot be attributed to some chance agency.” – (Reason, 1990)
• Use error (most relevant for medical device manufacturers):“act or omission of an act that has a different result
than intended by the manufacturer or expected by the operator.” – (IEC, 2004)
Knowledge-Based
Rule-Based
Skill-Based
Improvisation in unfamiliar environmentsNo routines or rules available to help handle
Protocolized behaviorProcess, Procedure
Automated RoutinesRequire little conscious attention
Figure adapted from: Embrey D. Understanding Human Behaviour and Error, Human Reliability AssociatesBased on Rasmussen’s SRK Model of cognitive control, adapted to explain error by Reason (1990, 2008)
SLIPS & LAPSES
1. MISAPPLY GOOD RULE
2. NOT APPLY GOOD RULE
3. APPLY BAD RULE
TRIAL & ERROR
What about
Violations?
Human-machine System
Task
Lars-Ola Bligård (2007) [adapted from Sanders and McCormick, 1993; Danielsson, 2001]
Where Does the Breakdown Occur?• Use errors occur because of a mismatch between the system user,
equipment, task and environment (FDA, 1999). • Kaye and Crowley (2000) described six reasons why use-related hazards
occur:1. Devices are used in ways that were not anticipated;2. Devices are used in ways that were anticipated, but inadequately controlled for;3. Device use requires physical, perceptual, or cognitive abilities that exceed those
of the user;4. Device use is inconsistent with user’s expectations or intuition about device
operation;5. The use environment affects device operation and this effect is not understood
by the user; or6. The user’s physical, perceptual, or cognitive capacities are exceeded when using
the device in a particular environment.
Defibrillator ExampleCardiac arrest1. nurse with patient2. charges unit…3. clears patient…4. presses “on” button5. Machine powers down
– 2-3 minute delay in shock
Hoyer, Christensen, et al. Annals of Emergency Medicine 2008; 52(5): 512-514.Fairbanks and Wears. Annals of Emergency Medicine 2008; 52(5): 519-521.
Error is Inevitable
• If error is inevitable… How to improve safety?– Reduce the occurrence of human error
• With better design• NOT training and policy
– Mitigate the effects of inevitable error• With better design
– Better feedback– Forcing functions
“Every system is designed to achieve exactly the results it gets.”
- Don Berwick, former IHI president, former director of CMS
Another Challenge!
“The single greatest impediment to error prevention in the medical industry is that we punish people for making mistakes.”
--Lucian Leape, Testimony to congress
Medical Devices and Software• More than 1/3 of medical device incidents involve
user error• More than 1/2 of device recalls for design problems
involve the user interface• Website/user interface design normally costs
upwards of 70% of total development costs• More than 1/2 of device recalls for design problems
involve the user interface
~B.R. Phillips – Mayo Clinic
Let’s Recap…So Far
• You can’t eliminate human error• Healthcare has a “name, blame,
shame, and train” culture• There are numerous
environments and users to design a single system for… and it’s changing
• Is the road ahead futile for medical device designers?
Usability
• ISO 9241-11:1998• The effectiveness, efficiency, and satisfaction with which
specified users achieve specified goals in particular environments
• The components are explained as follows:– Effectiveness: The accuracy and completeness with which specified
users can achieve specified goals in particular environments.– Efficiency: The resources expended in relation to the accuracy and
completeness of goals achieved.– Satisfaction: The comfort and the acceptability of the work system
to its users and other people affected by its use.
Usability engineering process (IEC 60601-1-6)
Usability – Early and Often!
Pre-MarketPost-Market
Design Requirements
Development Deployment
Cost of Intervention in Product Lifecycle
# Design Alternatives
$ of Design Changes
Implement usability early and reduce # of interventions and their associated costs!
Adapted from Bias & Mayhew, 1994, p. 80
Tailor Your Usability Approach
Formative Recommendations• Standards
– E.G. ANSI/AAMI HE75– ISO/IEC 62366:2007
• Card Sorting• Physical Ergonomics• Storyboard• Wireframe• Task Analysis
– Hierarchical task analysis– Cognitive task analysis
• Risk Analysis – FMEA
– Task Analysis; Failure Modes and their Effects; rate Severity, Occurrence, and Detection; calculate Risk Priority Numbers.
– When there isn’t a strong “design control” and your system relies on the user, ask yourself why?
American National Standards Institute & Association for the Advancement of Medical Instrumentation (2009). HE75: Human Factors engineering – Design of medical devices. Retrieved on July 25, 2013 from http://www.aami.org/publications/standards/he75.html
Formative Recommendations, cont..• Heuristic Evaluation– assess compliance of a
device or system with regard to human factors design principles and a predetermined set of design heuristics
• Heuristics– Consistency & Standards– Visibility of System State– Match Between System &
World– Minimalist– Minimize Memory Load– Informative Feedback– Flexibility & Efficiency– Good Error Messages– Prevent Errors– Clear Closure– Others… Heuristics - Nielsen, 1995
Formative Recommendations, cont..• Formative User Testing– Exploratory and is intended to find broad categories of use
challenges– 4-5 users per group find ~80% of the problems (Virzi, 1992)– Think aloud protocol– Design to collect performance measures (e.g. errors,
violations, time-on-task) AND subjective feedback (e.g. opinion ratings, design recommendations)
Virzi, R.. Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough? Human Factors, 1992, 34(4), 457 468
Summative Recommendations• Summative Evaluation
– Summative usability evaluation is the final validation intended to demonstrate that all risks have been identified and adequately addressed and that the impact of residual risks is minimized or acceptable. Usability evaluation of a complete, or near-complete, design under realistic conditions.
• Field User Events• Post-market Usability
Evaluations• Trade-off Studies
Regulatory Landscape
• Understand where your medical device fits here• Medical Device?– Class I General Controls– Class II General Controls and Special Controls– Class III General Controls and Premarket Approval
• Combination Product?• Mobile Medical App?
Medical Device• FDA Draft Human Factors Guidance - “The intent is to improve
the quality of the device user interface such that errors that occur during use of the device are either eliminated or reduced.”– Identify anticipated use-related hazards (derived analytically, see
Section 6) and unanticipated use-related hazards (derived through formative evaluations, see Section 7), and determine how hazardous use situations occur;
– Develop and apply strategies to mitigate or control use-related hazards (see Section 8); and
– Demonstrate safe and effective device use through human factors validation testing (see Section 10).
U.S. Department of Health and Human Services Food & Drug Administration Center for Drug Evaluation and Research (2011). Draft Guidance for Applying Human Factors and Usability Engineering to Optimize Medical Device Design. Report No. 1757. Retrieved on September 29, 2013 from: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM259760.pdf
Combination Product• Examples: drug/device, biologic/device,
drug/biologic, or drug/device/biologic• No HF/usability guidance available (expected 2014)• Human factors evaluation of training program
structure and materials is common– Important to show the incorporation of user feedback in
the design evolution of the device/instrument AND the training program
Mobile Medical App
• Final guidance available (not HF/Usability focused)• Intended environment policies for/against?• Multi-use device, not dedicated
chance of
unintended user
interactions
Cost Arguments and Evidence for Human Factors Integration, Issue 1 / October 2006 / HFI DTC
In Summary
• Error in healthcare is substantial• Medical device manufacturers can focus on reducing
“use errors” and mitigating their effects through human factors/usability
• Incorporate user feedback/testing early and often throughout the iterative design process– Result in a safer product; benefits cost, time, and scope
• Keep a record of the user-focused design efforts and tell your usability story in your regulatory submission
Andy Schaudt, M.S., M.B.A.Director of Usability Services
National Center for Human Factors in HealthcareMedStar Institute for [email protected]
www.MedicalHumanFactors.net202-244-9844