Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New...
-
Upload
informa-australia -
Category
Health & Medicine
-
view
237 -
download
0
description
Transcript of Dr Melissa Baysari, Research Fellow, Australian Institute of Health Innovation, University of New...
OPTIMISING DECISION
SUPPORT WITHIN EMMS
AN ONGOING CHALLENGE
MELISSA BAYSARI
+ JOHANNA WESTBROOK, RIC DAY, LING LI, KATE
RICHARDSON, ELIN LEHNBOM & MANY MORE
EMMS EFFECTIVENESS
Error rates fell by 66% at hospital A and 59.7% at hospital B
But most of the improvement was seen in “procedural” errors – e.g.
fewer incomplete orders
Little change in clinical errors (e.g. wrong doses) – those targeted by
decision support
DECISION SUPPORT (DS)
Can mean different things to different people
Computerised alerts
Pre-written orders
Reference material
What about:
Drop down lists?
Notes or instructions?
Calculators?
DS EFFECTIVENESS
Literature tells us that alerts can result in substantial
changes in prescribing behaviour
BUT
Most studies evaluate an alert for a specific condition or
problem
e.g. alerts designed to reduce the use of contraindicated drugs in
patients with renal failure drop in proportion of patients
receiving a contraindicated medication from 89% to 47%
Less evidence for the effectiveness of basic decision support
alerts within eMMS
e.g. few studies showing that DDI alerts lead to reductions in DDIs
aJAMIA 2005 12:269-74
ALERT FATIGUE
A consequence of too many alerts being presented
Main barrier to prescriber acceptance of computerised alerts
A significant problem for hospitals because it
results in user frustration & annoyance
leads to prescribers learning to ignore all alerts, even those that
present useful & sometimes safety critical information
Alert fatigue affects most doctors in most
organisations
most alerts are overridden
ALERTS @ SVH (MEDCHART)
Allergy
Therapeutic duplication
Dose range
Local messages
Pregnancy
50% alerts are for information only
10% prescribers must enter an
override reason
7 alerts do not allow prescriber to
continue
ALERT FATIGUE A PROBLEM?
1. Observations 2. Interviews 3. Chart audit
OPINIONS OF ALERTS
Registrar: It’s certainly helpful in, like I say, avoiding errors and
mistakes but I don’t think it really helps in deciding say what
antibiotic or what antihypertensive or whatever because that’s a
clinical decision
Registrar: The decision to prescribe something is based on your
clinical knowledge…by the time you type it in and prescribe it
you’ve already made that decision
Resident: I guess less words and more point forms would be
easier because then we wouldn’t have to scroll through
paragraphs and sentences of text
OPINIONS OF ALERTS
Registrar: It pops up so often which can be a very bad thing
because you’re dismissing it so often that you develop this sort of
mechanism so it can be bad in a sense that sometimes you might
miss some important things
Registrar: I at least scan them and work out what it is that they’re
trying to tell me. Often it’s saying you’ve just prescribed, do you
want to prescribe it again, and I’m like well yes, I do
Resident: I don’t have a problem with all the alerts because I
know what they say now before they even come up
CHART AUDIT
No reporting function in MedChart to allow us to extract alert
information - had to conduct a detailed audit of electronic
charts to identify alerts
Pharmacist randomly selected patients each day from a list
of all inpatients
180 medication charts reviewed (6 weeks)
The following info was recorded:
Patient info (MRN, age, sex)
# total active orders, # orders with 1 or more alerts
For orders with an alert: prescriber, med name, med schedule
(e.g. PRN), alert type
ONLY OVERRIDDEN ALERTS
Limitation: Only alerts that were overridden were visible on
charts for review
But our observational work showed that the proportion of
orders abandoned or changed is small (0-5%)
Organisations implementing eMMS should specify the
logging and reporting of alert data by vendors
RESULTS – PATIENTS & ORDERS
Mean patient age: 63.7 yrs (20-100 yrs)
58% patients were male
2209 orders were active
Mean: 12.3 orders/patient
96.8% of orders were initiated by junior doctors
RESULTS – ALERTS
600/2209 orders had 1 or more computerised alerts
27.2% of orders
934 alerts in total, mean 1.6 alerts/alerted order
Alert type # (% of total alerts)
Duplication 572 (61.2)
Local messages 232 (24.8)
Pregnancy 100 (10.7)
Allergy 21 (2.3)
Dose range 9 (1.0)
Total 934
PREGNANCY ALERTS
20 patients met the criteria (female, aged 12-55 yrs)
Of 119 meds ordered for these patients, 43.3% triggered a
pregnancy alert
Prescribers received on average 5 pregnancy alerts per
eligible patient (range 1-10 alerts)
½ the alerts for these eligible patients were pregnancy alerts
LOCAL MESSAGES
¼ alerts we found were local messages
Most offer prescribers advice rather than warning about a
safety critical event
Could these be removed and presented in a non-interruptive
fashion?
DUPLICATION ALERTS
Most frequent trigger = different drug, same therapeutic class
(40% of duplication alerts)
½ duplication alerts were triggered because a medication
was prescribed that was identical to, or in the same class as
a drug that had been ceased within the previous 24 h
During observations we noticed that a
number of duplication alerts were
being triggered because prescribers
were not using all the eMMS functions
PEOPLE USE SYSTEMS IN
UNEXPECTED WAYS
Most users of applications utilize only a sub-set of system
features
(Think of all the functions you DON’T use in Excel, on your
iPhone, on your washing machine…)
People use applications in less optimal ways to manage
problematic or poorly designed IT
E.g. users of an EHR used free-text boxes instead of the
appropriate functions because the functions were hard to find
STUDY AIM
To examine how the use of eMMS functions by prescribers can
influence alert generation
That is, to identify the proportion of duplication alerts triggered as
a result of prescribers not utilizing eMMS functions
SHORT-CUT IN EMMS
THEN, AND, OR = allow similar sequential, concurrent or
alternative orders for the same medication to be prescribed
together
e.g. Frusemide in the morning AND midday
+
These would save the prescriber up to 11 mouse clicks
During training, all Drs are shown where to find the short-
cuts and complete case scenarios using them
Drs are encouraged to use short-cuts as they save time
SHORT-CUT IN EMMS (2)
To make a change to an order on a patient’s chart, a doctor
should click on the order and edit the parameter (e.g. change
the dose), instead of ceasing and re-ordering the medication
PROCEDURE
For all orders where at least 1 alert was triggered we asked:
Could the use of a different system function (THEN, AND, OR,
or MODIFY) have prevented the alert from firing?
Yes = technically preventable
No = not technically preventable
PREVENTABLE ALERTS
189 alerts were technically preventable
= 1/3 of duplication alerts
= 20% of all alerts
Prescribers did not use the eMMS functions as intended,
despite the functions’ potential to improve efficiency of work
JAMIA 2012, 19: 1003-1010
WHY?
The efficient strategies are not known to users
and/or
The strategies are known but system design features are
poor
and/or
The strategies are not viewed as beneficial or consistent with
preferred prescribing practice
CONSISTENCY OR
EFFICIENCY?
There is a tension between designing systems which
replicate paper-based processes and integrate quickly into
clinical practice
vs.
Harnessing the advantage of technology to allow tasks to be
completed in more efficient ways, but which require a change
in work & cognitive processes
REDUCING ALERT FATIGUE
Following the discovery that too many alerts are being
presented, how do we decide what alerts to remove from the
system?
Previous study1:
Interviewed doctors & pharmacists
Found no alert types that all clinicians agreed could be turned off
Found specialties differed in the number and types of alerts they
thought could be safely turned off
1Van der Sijs et al. JAMIA. 2008;15(4):439-48.
THE DELPHI TECHNIQUE
Group facilitation technique used to obtain consensus
among experts in a systematic way
Consensus is reached by allowing participants to consider
their responses in light of the overall groups’ responses
Delphi previously used to:
Identify appropriate information to include in alerts
Determine what information about the user and context is helpful
in prioritizing and presenting alerts
STUDY AIM
To reach consensus among prescribers of different
specialties and with various levels of experience on
appropriate strategies for reducing alerts within eMMS
No previous studies have used Delphi for this purpose
Previous Delphi research has included recruitment of experts
in CPOE or decision support implementation, not users of
the system
SURVEY DEVELOPMENT
10-question web-based survey
Input was sought from prescribers, pharmacists & clinical
information system staff
In the survey, doctors were asked:
What alert types they found useful/not useful
What alert types, if any, they would remove from the system
To rate each alert type on a Likert scale of usefulness
Whether or not they believed 2 potential strategies for reducing
alerts numbers would compromise patient safety:
POTENTIAL STRATEGIES
Identified in our previous work on alert fatigue:
1. Modifying most local messages so that they were
presented as hyperlinks on the prescribing screen, rather
than interruptive alerts
2. Modifying therapeutic duplication alerts so that they fired
only when the initial order was active on a patient’s chart,
not when it was ceased within 24 hours
PROCEDURE
To recruit prescribers, an ad (with a link to the survey) was
posted in the weekly JMO bulletin sent to all JMOs at the site
In round 2, doctors were sent a personalized email
containing a link to their round 2 survey
Feedback about round 1 responses were incorporated
into each question in round 2:
SAMPLE ROUND 2 QUESTION
The percentages beside each option below indicate the proportion
of doctors who selected that option in round 1.
Q2. If you could remove only one alert type from the current alert
set in MedChart, which type would you remove?
In round 1, you selected ‘Pregnancy’.
☐ Allergy & intolerances (2%)
☐ Pregnancy (34%)
☐ Therapeutic duplication (28%)
☐ Local rule (13%)
☐ None, I’d not remove any alert type (23%)
CONSENSUS
Consensus was defined as 80% agreement between
participants on questions requiring a single response
Although consensus was not reached after 2 rounds, response
stability was apparent, making it unlikely that participants would
change views during a 3rd round
RESPONDENTS
Round 1: 47 prescribers, Round 2: 21 prescribers
Various specialties and levels of experience
Round 1
Alcohol and drug
Anesthetics
Cardiology
Clin Pharm
Dermatology
ED
Gastroenterology
Geriatrics
Surgery
Hematology
Immunology
ICU
Medical oncology
Nephrology
Neurology
Palliative care
Psychiatry
Rehabilitation
Respiratory
Urology
Night
shift/seconded
Round 2
Alcohol and drug
Cardiology
Clin Pharm
ED
Geriatrics
Surgery
Hematology
Immunology
ICU
Medical oncology
Neurology
Palliative care
Psychiatry
Rehabilitation
Night
shift/seconded
AREAS WHERE CONSENSUS
WAS REACHED
Prescribers agreed on what alert type should be retained
81% rated Allergy & intolerance alerts as the most useful alert
type
No participant believed this alert type should be removed
All participants rated this alert type as ‘often’ or ‘sometimes’ useful
Prescribers agreed that our suggested strategies would work
95% thought that changing local messages so they appeared as
hyperlinks on the prescribing screen would be safe
91% thought that changing duplication warnings so they only fired
when the initial order was active would be safe
AREAS WHERE NO
CONSENSUS WAS REACHED
0
10
20
30
40
50
Pro
port
ion o
f pre
scribers
Prescriber
responses to the
question ‘If you
could remove
one alert type
from the current
alert set in
MedChart, which
type would you
remove?’
ALERT USEFULNESS
0
10
20
30
40
50
60
Never Rarely Sometimes Often
Pro
port
ion o
f pre
scribers
Allergy
Pregnancy
Duplication
Local
Prescriber responses
to the question ‘How
useful is each alert
type in warning you
about prescribing
something potentially
dangerous for your
patients?’
STUDY CONCLUSIONS
We identified some strategies that users viewed as
appropriate for reducing alert numbers
1. Present local messages as hyperlinks
Not unexpected because many messages provide low
priority information
2. Ensure duplication alerts trigger when initial order is
active – this would eliminate more than ½ of these alerts
24 h time-frame is only useful for a small number of
medications (e.g. colchicine)
RESEARCH TRANSLATION
Based on observations, interviews and chart audit
Pregnancy alerts were removed
Many of the local messages were replaced
with corresponding pre-written orders
Next step – assess clinical impact of altering duplication
alerts so they fire only when the initial order is active
OTHER STRATEGIES
(not as easy to implement as they sound)
Tier alerts according to severity
Include only high severity alerts
Apply human factors principles in designing alerts
Customize alerts for doctors
CONCLUSIONS
Getting alerts right is a challenge
Most sensible approach: include only a few alert types and
provide alternative forms of DS to prescribers (e.g. pre-
written orders)
Continuously evaluate DS!
Quantitative and qualitative methods allow us to determine if DS
is working and why
Seeking input and feedback from users is invaluable