Clinical Decision SupportClinical Decision Support
Dr Jeremy Rogers MD MRCGPSenior Clinical Fellow in Health InformaticsNorthwest Institute of Bio-Health Informatics
Talk OutlineTalk Outline
Why we need it
What does ‘decision support’ mean ?
Work so far
Why we don’t use it
Talk OutlineTalk Outline
Why we need it
What does ‘decision support’ mean ?
Work so far
Why we don’t use it
Drivers for decision Drivers for decision supportsupport
► Growth of medical knowledge► Approx 100 articles were published in 1966 from RCTs;
► Over 10,000 annually by 1995 (Chassin, 1998)
► ‘The scarcely tolerable burden of information that is imposed taxes the memory but not the intellect’ (GMC 1993)
► Pressures to use knowledge► Evidence based medicine
► National service frameworks
► Clinical Governance
► Cost – e.g. $5.5M in 37 Days for one patient at Duke
► ‘Post genomic’ individualised medicine
Drivers for decision Drivers for decision supportsupport
► Public recognition of medical error► IOM “To err is human” (2000)
& “Crossing the quality chasm” (2001)
► More people die from medical errors than from breast cancer or AIDS or motor vehicle accidents
► Jessica Santillan case17 year old who had a heart and lung transplant from a donor with an incompatible blood group in Feb 2003 at Duke, and died after a re-do 13 days later
Committee on Quality of Committee on Quality of Health Care in AmericaHealth Care in AmericaUS Institute of Medicine : Quality Chasm Report, 2001US Institute of Medicine : Quality Chasm Report, 2001
(The American) health care delivery system is in (The American) health care delivery system is in need of fundamental changeneed of fundamental change
The current care systems cannot do the job The current care systems cannot do the job
Trying harder will not work Trying harder will not work
Changing systems of care willChanging systems of care will
Talk OutlineTalk Outline
Why we need it
What does ‘decision support’ mean ?
Work so far
Why we don’t use it
Kinds of decisionKinds of decision
Diagnosis
Intervention
Prognosis
Kinds of supportKinds of support
► Active vs Passive support► Making specific suggestions – one off, or continuing ?
► Critiqueing recorded actions – screw-up detection
► Tweaking / filtering information display
► Intelligent image processing
► Reminders ? Alerts ?
► Decision support, or decision making ?► Do we expect human to learn from device ?
Drowning in dataDrowning in dataThe case for DS in display filteringThe case for DS in display filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe36 yrsEngineerMarried, 2 children
12.10.96 Coryza: chest NAD: reassure13.10.96 URTI: wheezy: amoxycillin20.10.96 Anxiety: child admitted to H: reassure24.10.96 PEFR : 300 :10.11.96 PEFR : 400: CXR requested12.11.96 CXR Basal Consolidation: : erythromycin27.11.96 : Chest clear :07.03.97 Depression: death in family: paroxetine19.04.97 Gastoenteritis: : reassure01.06.97 : : rpt Rx paroxetine18.10.97 Sick note : :03.03.98 Viral URTI: PEFR 350: salbutamol04.03.98 WCC NAD : :30.06.98 PMR report : BP, ECG NAD :15.09.98 Eczema : : hydrocortisone05.11.98 Depression : : paroxetine03.01.99 Fibrositis: trigger spot lwr back: ibuprofen17.02.99 Allergic Asthma: PEFR 300: salbutamol21.03.99 Chest Inf: L base: erythromycin07.10.99 Med4: anxious :26.01.00 Asthma Review: :Repeat Rx Salbutamol
Encounters
Active ProblemsAsthma
Current MedicationSalbutamolHydrocortisone
PEFRBP WCC
This VisitCode Notes ActionPEFR 550 l /min
C/o Low Mood Declined antidepressantAsthma Influvac im BN #035679A4
Salbutamol inh 2 puff qds 1opChest NAD. No Problems.
Letters Results Appt
Drowning in dataDrowning in dataThe case for DS in display filteringThe case for DS in display filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe36 yrsEngineerMarried, 2 children
12.10.96 Coryza: chest NAD: reassure13.10.96 URTI: wheezy: amoxycillin20.10.96 Anxiety: child admitted to H: reassure24.10.96 PEFR : 300 :10.11.96 PEFR : 400: CXR requested12.11.96 CXR Basal Consolidation: : erythromycin27.11.96 : Chest clear :07.03.97 Depression: death in family: paroxetine19.04.97 Gastoenteritis: : reassure01.06.97 : : rpt Rx paroxetine18.10.97 Sick note : :03.03.98 Viral URTI: PEFR 350: salbutamol04.03.98 WCC NAD : :30.06.98 PMR report : BP, ECG NAD :15.09.98 Eczema : : hydrocortisone05.11.98 Depression : : paroxetine03.01.99 Fibrositis: trigger spot lwr back: ibuprofen17.02.99 Allergic Asthma: PEFR 300: salbutamol21.03.99 Chest Inf: L base: erythromycin07.10.99 Med4: anxious :26.01.00 Asthma Review: :Repeat Rx Salbutamol
Encounters
Active ProblemsAsthma
Current MedicationSalbutamolHydrocortisone
PEFRBP WCC
This VisitCode Notes ActionPEFR 550 l /min
C/o Low Mood Declined antidepressantAsthma Influvac im BN #035679A4
Salbutamol inh 2 puff qds 1opChest NAD. No Problems.
Letters Results Appt
Goal of supportGoal of support
► Influence outcome► Good things more likely; bad things less likely
► Outcomes…► Fatal events are only the tip of the iceberg
►Easiest to measure, and most dramatic, but….
► Non fatal events
►Side effects
►Sub-optimal treatment
►Inappropriate treatment
► Non harmful events
►Inefficiency & Confusion
►Inappropriate resource consumption
►Bed stay
►Repeated re-investigation
Kinds of DS technologyKinds of DS technology
► Statistical► ‘93.467% of the time, things that quack
and have webbed feet are ducks”
► Model-based► ‘It’s definitely a duck because
you told me its mother was a duck’
► Neural Networks► ‘Of all the things you’ve shown me so far,
it looks most like the ones you said were ducks.’
Talk OutlineTalk Outline
Why we need it
What does ‘decision support’ mean ?
Work so far
Why we don’t use it
The Story so far…The Story so far…
“Three decades of research into computer aids for medical decision making
have resulted in thousands of systems and a growing number of successful clinical trials…”
BMJ 1997;315:891 (4 October)
Projects past and presentProjects past and present
► Acute Abdominal Pain (1972)
► Mycin (1977)
► Internist/QMR (1980s)
► DXPlain
► ILIAD
► Sophie
► Medical Logic Modules (Arden Syntax) (1989)
► ProFORMA
► Protégé
► PRODIGY (1997-)
► Prescribing Indicators
► Isobel (2000)
► NHS Direct (2000)
► Knowledge Coupling (PKC.com)
1970s & 1980s1970s & 1980sBehold, the Oracle…Behold, the Oracle…
Mycin – Shortliffe 1970s, Mycin – Shortliffe 1970s, StanfordStanford
> (mycin)
------ PATIENT-1 ------
Patient's name: Sylvia Fischer
Sex: female
Age: 27
------ CULTURE-1 ------
From what site was the specimen for CULTURE-1 taken? blood
How many days ago was this culture (CULTURE-1) obtained? 3
------ ORGANISM-1 ------
Enter the identity (genus) of ORGANISM-1: unknown
The gram stain of ORGANISM-1: ?
A GRAM must be of type (MEMBER ACID-FAST POS NEG)
The gram stain of ORGANISM-1: neg
Is ORGANISM-1 a rod or coccus (etc.): rod
What is the AEROBICITY of ORGANISM-1? why
It is known that:
1) THE GRAM OF THE ORGANISM IS NEG
2) THE MORPHOLOGY OF THE ORGANISM IS ROD
Therefore,
Rule 107:
If
1) THE AEROBICITY OF THE ORGANISM IS AEROBIC
Then there is suggestive evidence (0.8) that
1) THE IDENTITY OF THE ORGANISM IS ENTEROBACTERIACEAE
What is the AEROBICITY of ORGANISM-1? aerobic
Is Sylvia Fischer a compromised host? yes
Is Sylvia Fischer a burn patient? If so, mild or serious? why
It is known that:
1) THE SITE OF THE CULTURE IS BLOOD
2) THE GRAM OF THE ORGANISM IS NEG
3) THE MORPHOLOGY OF THE ORGANISM IS ROD
Therefore,
Rule 52:
If
1) THE BURN OF THE PATIENT IS SERIOUS
Then there is weakly suggestive evidence (0.4) that
1) THE IDENTITY OF THE ORGANISM IS PSEUDOMONAS
Is Sylvia Fischer a burn patient? If so, mild or serious? serious
Findings for ORGANISM-1:
IDENTITY: ENTEROBACTERIACEAE (0.800) PSEUDOMONAS (0.760)
Is there another ORGANISM? (Y or N) Y
------ ORGANISM-2 ------
Enter the identity (genus) of ORGANISM-2: unknown
The gram stain of ORGANISM-2: (neg .8 pos .2)
Is ORGANISM-2 a rod or coccus (etc.): rod
What is the AEROBICITY of ORGANISM-2? anaerobic
Findings for ORGANISM-2:
IDENTITY: BACTEROIDES (0.720) PSEUDOMONAS (0.646)
Abdominal Pain: Abdominal Pain: De Dombal De Dombal (1972)(1972)
A multicentre study of computer aided diagnosis for patients with acute abdominal pain was performed in eight centres with over 250 participating doctors and 16,737 patients.
Performance in diagnosis and decision making was compared over two periods: a test period (when a small computer system was provided to aid diagnosis) and a baseline period (before the system was installed). The two periods were well matched for type of case and rate of accrual.
The system proved reliable and was used in 75.1% of possible cases.
User reaction was broadly favourable.
Abdominal Pain: Abdominal Pain: De DombalDe Dombal
During the test period improvements were noted in diagnosis, decision making, and patient outcome.
Initial diagnostic accuracy rose from 45.6% to 65.3%. The negative laparotomy rate fell by almost half, as did the perforation rate among patients with appendicitis (from 23.7% to 11.5%). The bad management error rate fell from 0.9% to 0.2%, and the observed mortality fell by 22.0%.
The savings made were estimated as amounting to 278 laparotomies and 8,516 bed nights during the trial period--equivalent throughout the National Health Service to annual savings in resources worth over 20m pounds and direct cost savings of over 5m pounds. Computer aided diagnosis is a useful system for improving diagnosis and encouraging better clinical practice. Br Med J (Clin Res Ed) 1986 Sep 27;293(6550):800-4
Medical Logic ModulesMedical Logic Modules(Arden Syntax)(Arden Syntax)
maintenance:
title: ;;
filename: template;;
version: 1.00;;
institution: ;;
author: ;;
specialist: ;;
date: 1993-01-01;;
validation: testing;;
library:
purpose: ;;
explanation: ;;
keywords: ;;
citations: ;;
knowledge: type: data-driven;;
data: ;;
evoke: ;;
logic: ;;
action: ;;
end:
An MLM…An MLM…
maintenance:
title: Check for adequacy of therapeutic anticoagulation with warfarin;;filename: warfarin_anticoagulation;;version: 1.07;;institution: Columbia-Presbyterian Medical Center;;author: Randolph C. Barrows, Jr., MD ([email protected]);;specialist: ;;date: 1994-04-28;;validation: testing;;
library:
purpose: To warn the health care provider that a patient maintained on warfarin is NOT in a therapeutic range for low-intensity or full-intensity anticoagulation. Low-intensity anticoagulation is defined as a prothrombin INR in the range of 2.00 - 3.00 (roughly corresponding to a PT in the range of 1.2-1.5 times control). Full-intensity anticoagulation is defined as an INR in the rage of 3.00 - 4.50 (roughly corresponding to a PT in the rage of 1.5 - 2.0 times control).;;
explanation: ;;
keywords: ;;
citations: Scientific American Medicine;;
……and (some of) its logicand (some of) its logic
/* the INR-containing procedures */
storage_of_INR := EVENT {
'32506~service event', ‘2256~presbyterian coagulation profile'; '32506~service event', ‘2302~stat coagulation profile' };
/* See if patient has a warfarin order. Probably need to add 31058 Bishydroxycoumarin Preparations Here I only want header table info, no components. Is it ok to say null components? */
(start_time, status, order_key, frequency):= READ LAST {
'dam'="PDQORD1", display_header'="TRSKF",'display_comp'=""; ; '28612~CPMC Drug: Coumadin 10 Mg Tab', '28613~CPMC Drug: Coumadin 2 Mg Tab', '28614~CPMC Drug: Coumadin 2.5 Mg Tab', '28615~CPMC Drug: Coumadin 5 Mg Tab', '29932~CPMC Drug: Ud Coumadin 10 Mg Tab','29933~CPMC Drug: Ud Coumadin 2 Mg Tab', '29934~CPMC Drug: Ud Coumadin 2.5 Mg Tab', '29935~CPMC Drug: Ud Coumadin 5 Mg Tab','33033~CPMC Drug: Coumadin 7.5 Mg Tab' };
Knowledge Couplers: Knowledge Couplers: PKC.comPKC.com
Larry Weed MD
Some CPOE Success StoriesSome CPOE Success Stories
► Barnes-Jewish Hospital, St. Louis, Missouri130 potentially dangerous drug interactions identifiedtwo-thirds of those involving the drug cisapride averted
► Brigham and Women’s Hospital, Boston81% decline in medical errors after implementation CPOE64% of decline due to first, and simplest, version of the technology, which included features such as predetermined lists of medications and doses, display of patient data, basic drug dosage, interaction, and duplication checking.
► Montefiore Medical Center, New York City50% decrease in medication errors following CPOETime from placing an order to its arrival in pharmacy reduced to two hours.
► Ohio State University Medical Center, Columbus, OhioLength of stay decreased by two days following CPOEPharmacy orders turnaround reduced by two hoursPharmacy charges per admission reduced by $910
► University Community Hospital, Tampa, Florida77% reduction in all adverse drug events, and 85% in severe ADEs Cost of drugs for one family reduced by more than $200,000 per year.
► Children’s Hospital of Pittsburgh50% reduction in harmful errorVirtual elimination of weight-related adverse drug events Complete eradication of transcription/handwriting errors 50% reduction in medication delivery times.
Other successes…Other successes…
► Strong evidence suggests that some CDSSs can improve physician performance. Additional well-designed studies are needed to assess their effects and cost-effectiveness, especially on patient outcomes(Johnston 1994)
► Mothers receiving computer-generated reminders had 25% higher on-time immunization rate for their infants (Alemi, 1996)
► Decision support system was safe and effective and improved the quality of initiation and control of warfarin treatment by trainee doctors(BMJ 1997;314:1252)
► Computerized physician order-entry reduced adverse drug events by 55% (Bates, 1998)
► 9% of redundant lab tests at a hospital could be eliminated using a computerized system (Bates, 1998)
► 74% of the studies of preventive healthcare reminder systems and 60% of the evaluations of drug dosing models reported a positive impact(Trowbridge & Weingarten, AHRQ, 2001)
..and some failures..and some failures
► (PRODIGY) - No effect was found … on the management of asthma or angina in adults in primary careBMJ 2002; 325: 941-944
► ..decision support system did not confer any benefit in absolute risk reduction or blood pressure control BMJ 2000;320:686-690
► Computerised decision support systems have great potential for primary care but have largely failed to live up to their promiseBMJ 1999;319:1281
My own failure: My own failure: Prescribing IndicatorsPrescribing Indicators
► General Practice Repeat Prescribing► Patients get more drug without seeing doctor
►typically, enough for 1-3 months
► 35% of population at any one time on repeat Rx
► Medication Review► Accepted part of good clinical practice
► Requirement in NSF for Older People
► But: signing authorities is daily batch process
►>30 scrips per GP per day
►No time for careful review
What is ‘Medication Review’ ?What is ‘Medication Review’ ? Indicators of ‘quality’ Indicators of ‘quality’ prescribingprescribing
► Cantrill et al: 13 indicators:► Dose too high or too low?
► Course too long ?
► Expensive or useless drug ?
► Interaction with another drug ?
► Contraindicated ?
► By brand ?
► REASON FOR USE DOCUMENTED ?
► Manual system: impractical
► Our project: (2000-2002) ► computerise the indicators
Complex implementation..Complex implementation..
Indication Code Rubric
Atrial fibrillation 14AN. H/O atrial fibrillation
3272. ECG: atrial fibrillation
3273. ECG: atrial flutter
7936A IV pacer control of A Fib
G573. Atrial fibrillation / flutter
Patient ID: 4578Medication: DITA906 DISR10514BProblem List: 183... (Oedema) 1B17..
(Depressed) G5732. (Paroxysmal Atrial fibrillation) G73z0. (Intermittent claudication) H3.... (Chronic obstructive pulm.dis.) 137S.. (Ex smoker) 246... (O/E - blood pressure reading) 442... (Thyroid hormone tests) 44P... (Serum cholesterol) 7L172. (Blood withdrawal for testing)
Ontology ID Product Rubric
345031(oral dig) DITA905 Digoxin 125 mcg tab
345031 DITA906 Digoxin 250 mcg tab
345031 DITA908 Digoxin 62.5 mcg tab
IDENT “9099269”
MAIN digoxin
PROPERTIES
HAS_DRUG_FEATURE physiological action
WHICH_IS process
ACTS_ON heart
HAS_DRUG_FEATURE indication
FOR treating
ACTS_ON supraventricular arrhythmia
HAS_DRUG_FEATURE indication
FOR treating
ACTS_ON atrial fibrillation
HAS_DRUG_FEATURE information source
IS_PART_OF interaction appendix
345031Oral
Digoxin tablet
305075Digoxin injection
305084Digoxin Liquid
329308Digoxin elixir
305093Digoxin Paed inj
9099269Systemic Digoxin
G57.. Cardiac dysrhythmias
G573. Atrial fibrillation and flutter
G5730 Atrial fibrillation
G5731 Atrial flutter
G5732 Paroxysmal atrial fibrillation
G573z Atrial fibrillation and flutter NOS
..and disappointing ..and disappointing resultsresults
► Machine says there is no recorded indication in 33% of prescribing events
► BUT high false positive rate: 62%► => it is wrong, most of the time
► Why ?
Of all alerts where machine says Of all alerts where machine says ‘no indication’…‘no indication’…
Mapping error27%
No Record38%
Human could infer3%
Idiosyncratic record
27%
BNF Omits5%
Problems with the Problems with the oracleoracle
► Painful data acquisition► Exhaustive
► Includes exhaustive negative findings
►(which clinicians traditionally largely omit)
► Slow to use
► Poor support for clinical workflow
► Clinician is passive
► Infrequent recognised need
1990s –1990s –More modest aspirationsMore modest aspirations
► Narrow Domain systems► ECG interpretations
► Arterial blood gas interpretation
► Predicting drug-drug interaction
► Alerts and Reminders► Out of range test flagging
► But plans for the oracle are resurfacing in expectation of imminent EPR
Talk OutlineTalk Outline
Why we need it
What does ‘decision support’ mean ?
Work so far
Why we don’t use it
You can lead a horse to You can lead a horse to water…water…
“Three decades of research into computer aids for medical decision making have
resulted in thousands of systems and a growing number of successful clinical
trials…”
“Yet only a handful of applications are in everyday use”
BMJ 1997;315:891 (4 October)
QMR Diagnostic decision-support system for internists 1972 routine use
PUFF Pulmonary function tests 1977 ?
HELP Knowledge-based HIS 1980 ?routine use
ACORN Coronary care admission 1987 decommissioned
DXplain Clinical decision support 1987 routine use
Liporap Dyslipoproteinaemia phenotyping 1987 ?routine use
MDDB Diagnosis of dysmorphic syndromes 1988 ?routine use
Epileptologists' Assistant Nurse progress note assistant 1989 decommissioned
Cancer, Me? Patient cancer advice 1989 ?
Hepaxpert I, II, III Hepatitis serology 1989 routine use
Interpretation of acid-base disorders acid-base disorders 1989 ?routine use
Managed Second Surgical Opinion System Managed care 1989 ?
Colorado Medicaid Utilization Review System Prescription quality review 1990 ?
Geriatric Discharge Planning System Patient discharge planning 1990 ?
Microbiology/ Pharmacy Expert System Drug sensitivity 1991 ?routine use
PEIRS Pathology reports 1991 decommissioned
NéoGanesh Ventilator manager 1992 2001
POEMS Post-operative care 1992 ?
SETH Clinical toxicology 1992 ?routine use
Jeremiah Orthodontic treatment planner 1992 ?routine use
Clinical Event Monitor Clinical alerts 1992 ?routine use
VIE-PNN Neo-natal parentral nutrition 1993 ?In use
CEMS Mental health decision support system 1993 routine use
GermAlert Infection control 1993 ?routine use
Germwatcher Infection control 1993 ?routine use
Orthoplanner Orthodontic treatment planner 1994 ?routine use
RaPiD Designs removable partial dentures 1994 ?routine use
DoseChecker Drug dose checker 1994 ?routine use
Coulter® FACULTYTM Haematology 1995 ?routine use
SahmAlert Drug sensitivity 1995 ?routine use
Reportable Diseases Infection control 1995 ?routine use
TxDENT Screeing dental patients 1997 ?routine use
RetroGram Decision support for drug regimens for HIV-infected patients 1999 routine use
Automedon Ventilator manager 2001 routine use
ERA Web-enabled electronic decision support and referrals system for cancer 2001 Under evaluation
Therapy Edge Web-enabled decision support system for the treatment of HIV 2001 routine use
ATHENA DSS for the management of hypertension in primary care 2002 routine use
Decision Support Systems in Use Today (2003)
http://www.openclinical.org/aisinpractice.html
Why ? – the domainWhy ? – the domain
► Rigid criteria difficult to apply in chaotic settings
► Medical data doesn't fit quantised definitions► Even complex decision support algorithms require simplified
and standardised inputs by users
► And descriptive data is very hard to quantise
► Rules are situation specific► localising decisions to available resource is costly
► When are decisions actually made ?► To be effective, system needs to be physically
available in situation where decision is made
Why ? - the technologyWhy ? - the technology
► Highly mobile workforce vs highly static computers
► Slow computers
► Crude knowledge bases poor performance► Lack of stats for bayesian approaches
► Crude KR technology for model-based
► Closed software architectures► Can’t integrate 3rd party DS modules with EPR
Why ? – the lawWhy ? – the law
► Medicolegal aspect of EPR► Confidentiality & Consent
► HIPAA
► Medicolegal aspects of DS technology► Responsibility for action rests with clinician
► Systems that are as effective as clinician overall no help if behaviour includes obvious clinical howlers
► Burden of recording why did not follow DS advice
Why ? – the peopleWhy ? – the people
► Poor data quality
► Numerical data easy to obtain
► Much of medicine not numerical
► Inconsistent data entry
Data QualityData Quality((Frequency of recording per GP per year)Frequency of recording per GP per year)
READ CODE Practice A Practice B
Sore Throat Symptom 0.6 117
Visual Acuity 0.4 644
ECG General 2.2 300
Ovary/Broad Ligament Op 7.8 809
Specific Viral Infections 1.4 556
Alcohol Consumption 0 106
H/O Resp Disease 0 26
Full Blood Count 0 838
Why? – the peopleWhy? – the people
► Poor data quality
► Numerical data easy to obtain
► Much of medicine not numerical
► Inconsistent data entry
► What happened to my clinical autonomy ?
► Interface issuesBMJ 1999;318:1527-1531
► I know what I’m doing► Perception of infallibility
► 88% of the time users requested to bypass PRODIGY (Beaumont 1988)
► Reluctance to change clinical practice to fit the tool
► Weed’s knowledge couplers
► Users intolerant of less than perfect performanceBMJ 2003;326:314
Why ? - moneyWhy ? - money
Through more improved choice of initial antibiotics to treat pneumonia, a group of mid-west hospitals decreased complications, mortality rates and hospital days and costs…
Improved management of diabetic patients through frequent e-mail communication can produce better outcomes and fewer visits…
…but hospital revenues also decreased as patients shifted from higher paying to lower paying DRGs.
…but lower physician group revenues under ‘fee for service’ payment.
SummarySummary
► Research and commercial products pre-date IOM by almost 30 years
► Widespread adoption has not occurred even where results were positive
► Significant hurdles remain► Legal
► Technical - EPR is harder than it looks
► Human factors
Top Related