Post on 13-Jan-2016
Usability & Human Factors
Decision Support Systems: a Human Factors Approach
This material (Comp15_Unit7) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.
Decision Support Systems: a Human Factors Approach
Learning Objectives
2
• Discuss factors in understanding Human Decision Making
• Discuss Clinical Decision Support Systems (CDSS)
• Discuss computer provider order entry systems (CPOES) and clinical decision support systems (CDSS)
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Patient Safety
3Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Human Factors Approach
4Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Understanding Decisions
5Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Medical Decision Making Research
6Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Heuristics and Biases
7Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Hindsight Bias
8Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Hindsight Bias: So What?
9Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Confirmation Bias
10Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
The Cost of Confirmation Bias
11Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Classic DM Problem (Eddy, 1982)
• Estimate the probability that a woman has breast cancer given that she has a positive mammogram on the basis of the following information:
– The probability that a patient has breast cancer is 1%. (This provides the prior probability)
– If the patient has breast cancer, the probability that the radiologist will correctly diagnose it is 80% (This provides the sensitivity or hit rate)
– If the patient has a benign lesion (no breast cancer), the probability that the radiologist will misdiagnose it is 9.6% (This provides the false positive rate).
• What is the probability that a patient with a positive mammogram actually has breast cancer?
• Probability of breast cancer is only 7.8%, while Eddy reports that 95 out of 100
• 95 out of 100 doctors estimated this probability to be between greater than 75%
12Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Framing Effect
13Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Survival vs. Mortality
• McNeil et al (1982) presented a hypothetical lung cancer decision scenario to physicians and patients– The treatment options were radiation therapy, which had an immediate
higher survival (lower mortality) rate, but a lower 5 year survival rate. – Frame 1: treatments were described in terms of survival rates – Frame 2: treatments were described in terms of mortality rates
• Results: – Survival frame, clear preference for surgery, – Mortality frame, the two choices were preferred almost equally.
• One possible explanation is that the positive framing leads to more risk averse choices, while the negative framing increases risk-seeking decision making
14Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
DM in Naturalistic Settings
15Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Decision Support Systems
16Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Clinical Decision Support Systems
17Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Star Trek Tricorder: The Ultimate Clinical Decision Support Tool
http://www.flickr.com/photos/bojo/4078685614/lightbox/#/
18Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Forms of CDSS Advice
19Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
The Case for Clinical Decision Support
20Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Degrees of CDSS Computerization
21Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Degrees of Computerization Continued (6-10)
22Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Computerized Provider Order Entry Systems (CPOE)
23Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Promise of Order-Entry Systems
24Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Some Advantages of CPOE Systems
25Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
A Drug-Drug Interaction Scenario
• “When ordering a new medication, a prescriber may not be aware that two drugs interact, or may not be keeping in mind the other medications that the patient is taking.
• As an example, consider the case of a hospitalized patient who is being treated with venlafaxine (Effexor) for chronic depression and develops an infection with a drug resistant bacterium requiring treatment with linezolid, a new antimicrobial agent.
• The interaction between linezolid and venlafaxine (serotonin syndrome -- altered mental status, including agitation, confusion and coma, neuromuscular hyperactivity, and autonomic dysfunction) is very severe but may not be known to the practitioner.
• While writing the order for linezolid, an alert screen can warn the practitioner that these two drugs should not be used together.
• The alert screen may offer the prescriber the opportunity to cancel the order, to discontinue
• the existing medication that interacts with the newly ordered medication, or to order a test that could detect the interaction or monitor therapy. The alert screen may prompt the physician to have a conversation with the patient regarding potential side effects of the medications. Any of these consequences of the decision support software could be beneficial.”
Kuperman , G., et al, (2007).
26Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Challenges with Order Entry
27Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
CPOE Paradox
28Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Cognitive Evaluation of Interaction with a CDSS
29Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Weight-Based Heparin Ordering
30Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Methods
31Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
CPOE Screen
Horsky, J., Kaufman, D. R., & Patel, V. L. (2005).
32Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Decision Support Alert Weight-based IV Heparin Protocols
Horsky, J., Kaufman, D. R., & Patel, V. L. (2005).
33Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Results - Presentation Salience
34Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Results - User Behavior
35Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Summary
36Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Role of CPOE Systems in Facilitating Medical Errors
37Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Information Errors: Fragmentation and Systems Integration Failure 1
38Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Information Errors: Fragmentation and Systems Integration Failure 2
39Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Human-Machine Interface Flaws
40Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Automation Bias
41Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Anti-Automation Bias
• Errors of dismissal, where computer advice is ignored
• Clinicians routinely disable or ignore the alarms or alerts on clinical monitoring devices – Legitimate reasons such as high false alarm
rates [or repetition of the same alarms – Less valid reasons such as not wanting to be
interrupted
42Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Barriers to Prescriber Decision-Making and Clinical Workflow
43Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
15 Barriers to Prescriber Decision-Making
BARRIER DESCRIPTION
1. Poor Screen Display
Alert display does not support alert resolution and/or prescriber workflow
2. Inadequate Alert Specification
Alert does not provide information on why it was triggered and/or the potential problem
3. Actual or Perceived Lack of Evidence
Alert is not evidence-based, does not provide a reference to evidence that does exist
4. Unclear Level of Risk
Alert does not provide clear information on relative risk of harm for a given patient
5. Redundancy Repeated alerts within the same encounter or over multiple encounters for a given patient
1.1 Table: (Kaufman, 2010).
44Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Barriers Continued 6. Low Alert Signal to Noise Ratio
Numerousness of alerts leads to information overload, prescriber desensitization, and potential for missing key alerts
7. Inadequate Allergy Logic
Alert system does not distinguish between true allergies and bothersome, but non-serious, side effects
8. Duplicate Workload
Alert duplicates other required work processes
9. Paper Prescriptions & Limited CPOE
Some medications are not or cannot be entered electronically, and therefore, are not reviewed by the alert system
10. Unclear System Capabilities
Alert system does not adequately reveal its capabilities/limitations to the prescriber; full functionality of the alert system is ambiguous
1.2 Table: (Kaufman, 2010).
45Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Barriers Elaborated
46Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Human Factors and Information Management
47Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Situation Awareness
48Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Mental Workload
49Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
CPOE/CDSS Design Recommendations
50Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Designing for Better Workflow
51Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Decision Support Systems: a Human Factors Approach
Summary
• Computer-based decision support systems offer great promise for reduction of errors in medicine and facilitation of quality patient care
• Results on system efficacy have thus far been equivocal– CDSS can lead to new types of errors
• Adherence to usability/human factors principles can lead to superior design and enhanced performance
52Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Decision Support Systems: a Human Factors Approach
ReferencesReferences:
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., et al. (2007). Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc, 14(1), 29-40.
Coiera E, Westbrook J, Wyatt J. The safety and quality of decision support systems. Methods Inf Med. 2006;45 Suppl 1(suppl 1):20–5.me06010020
Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 249-267). Cambridge, England: Cambridge University Press.
Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354.
Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.
J.A. Osheroff, J.M. Teich, B. Middleton, E.B. Steen, A. Wright and D.E. Detmer, A roadmap for national action on clinical decision support, J Am Med Inform Assoc 14 (2) (2007), pp. 141–145.
Karsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. AHRQ Publication No. 09-0054EF; Rockville (MD): Agency for Healthcare Research and Quality; June 2009.
Khajouei R, Jaspers MW: CPOE system design aspects and their qualitative effect on usability. Stud Health Technol Inform 2008 , 136:309-14
53Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach
Decision Support Systems: a Human Factors Approach
ReferencesReferences:
Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized physician order entry systems in facilitating medication errors. JAMA, 293(10), 1197-1203.
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., et al. (2007). Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc, 14(1), 29-40.
McNeil, B. J., Pauker, S. G., Sox, H. C., Jr., & Tversky, A. (1982). On the elicitation of preferences for alternative therapies. N Engl J Med, 306(21), 1259-1262.
Reckmann, M. H., Westbrook, J. I., Koh, Y., Lo, C., & Day, R. O. (2009). Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc, 16(5), 613-623.
Russ, A. L., Zillich, A. J., McManus, M. S., Doebbeling, B. N., & Saleem, J. J. (2009). A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc, 2009, 548-552.
Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA, Panel CDSER. Clinical decision support in electronic prescribing: Recommendations and an action plan. J Am Med Inform Assoc 2005;12(4):365-76.
Images:
Slide 18: Retreived on October 4th, 2010 form http://www.flickr.com/photos/bojo/4078685614/lightbox/#/
Slide 32: Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354
Slide 33: Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354
54Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability & Human Factors Decision Support Systems: A Human Factors Approach