Sepsis Biosurveillance at Dignity Health: Our Process Improvement Story

March 3, 2016 Department of Medical informatics and Data Science:

Alyson D’Andrea, RN MMI Ken Ferrell, Data Scientist / Biostatistician

Conflict of Interest Alyson D’Andrea, RN MMI Ken Ferrell, Data Scientist / Biostatistician Have no real or apparent conflicts of interest to report.

Agenda • Our Story • Statistical Validation • End User Workflow • Outcomes • Unexpected Learning

Learning Objectives • Describe system and workflow configuration to monitor for and report

potential SIRS and sepsis • Discuss attribution decisions for providers and nursing • Display our dashboard for performance monitoring • Discuss enterprise analytics hub technology and our ability to scale

Our program leverages Big Data and the world’s best analytic tools in a secured cloud-environment to improve health outcomes and reduce

costs. We use data across the continuum of care to improve outcomes of patients, increase knowledge worker productivity, enable self-service access,

and markedly increase calculation speed and insights to decision makers.

Dignity Health Insights™ Vision Statement

1. Simplify Data Integration Across the Extended Enterprise 2. Manage the Financial Risks and Incentives of Emerging Reimbursement Models 3. Proactively Improve Care Quality and Outcomes 4. Drive Greater Efficiency of Care Delivery 5. Engage Patients as Unique Individuals

Evidence Generating Medicine

Center of Excellence Platform The heart is an Hadoop platform which decreases storage cost by 90%, aggregates data sources, enables enhanced security and the flexibility to answer unanticipated questions

Sepsis Biosurveillance Program • Goals:

– Save 400 lives/year – Reduce ICU length of stay by 10% – Reduce costs

• Strategy – Identify Key Performance Indicators (KPI’s) – Implement Cerner’s sepsis algorithm (“St. John Sepsis Agent” – Create feed back loop

Statistics

A visual comparison of normal and paranormal distributions Matthew Freeman. J Epidemiol Community Health 2006;60:6

Statistical Validation • Validation should address the following questions:

– How often will the St. John’s Agent fire? – Where will the St. John’s Agent fire? – How accurate is the St. John’s Agent? – Is the St. John’s Agent configured properly?

Statistical Validation • Possible validation strategies:

– Use existing studies – Engage facilities that currently use the St. John’s Sepsis Agent – Rely on Cerner to provide statistics – Perform an evaluation on our patient populations

“Silent Mode” Evaluation 1. Enable St. John’s Agent in “Silent Mode” 2. Collect data for 3 months (Aug-Oct 2014) 3. Correlate agent activity with discharge diagnoses 4. Coordinate manual chart review

• All FN, FP • Sample of TP, TN

5. Prepare alert statistics • Sensitivity, Specificity, Relative Risk • Frequency of alerts • What clinical triggers are firing alerts

6. Review results with stakeholders

Frequency Evaluation

Location Location Type

SIRS Alerts Sepsis Alerts SIRS + Sepsis Alerts

90 Day Total Daily Average 90 Day Total Daily Average 90 Day Total Daily Average ED Emergency 161 1.79 160 1.78 321 3.57

CCU1 ICU 48 0.53 74 0.82 122 1.36 EDIP Emergency 44 0.49 61 0.68 105 1.17 CCU2 ICU 32 0.36 57 0.63 89 0.99 TEL2 Floor 27 0.3 42 0.47 69 0.77 TEL1 Floor 24 0.27 30 0.33 54 0.6

MSG3 Floor 16 0.18 21 0.23 37 0.41 MSG2 Floor 13 0.14 11 0.12 24 0.27 MCH Floor 7 0.08 14 0.16 21 0.23

MSG1 Floor 6 0.07 11 0.12 17 0.19 SDCI Floor 1 0.01 0 0 1 0.01

MEC1 Floor 0 0 1 0.01 1 0.01 Total: 379 4.21 482 5.36 861 9.57

Performance Evaluation

Pilot Facility #1 Alert No Alert Total Sepsis Dx TP = 397 FN = 80 477

No Sepsis Dx FP = 264 TN = 4,299 4,563 Total 661 4,379 5,340

Example of the evaluation performed once data was enriched by manual chart review.

• Sensitivity = 0.832 • Specificity = 0.942 • Relative Risk = 32.9

Findings • Common FN findings:

– Due to latency in entry of vitals – “Close calls” – Potential organ dysfunction tests not ordered

• 20-30% of FP identified as TP by manual chart review personnel – Nearly all had primary diagnosis of infection – Treatment pattern indicative of sepsis prophylaxis

• Sensitivity ranges between 75-92% at facility level • Specificity ranges between 87-95% at facility level • Relative Risk of alerted patient between 20-100 • Several missing configuration elements discovered

More Findings

• 80-90% of patients receiving sepsis alert in ED were admitted – Excludes AMA/LWBC/Transfers

• 30% of patients receiving a SIRS alert will receive a sepsis alert – Median time between alerts = 5 hours

• SIRS and Sepsis alerts occur in roughly equal numbers* • 35-50% of SIRS and Sepsis alerts occur in ED* • 15-30% of SIRS and Sepsis alerts occur in ICU*

* - Vary by facility

End User Workflow Cost LOS

Focus our smarter alert • To the “right” person

– Primary Nurse, Charge Nurse, Triage Nurse – Attending Provider

• At the “right” time – As SIRS or sepsis is identified (documented) – 24/48 hours after the last alert

Configure the Workflow

Create the Feedback Loop • Monitor workflow performance • Assign a goal owner for KPI’s • Nursing attribution:

– 1. The nurse who documented alert notification – 2. The nurse who documented the shift screen

• Provider attribution: – 1. The provider that was notified – 2. Attending provider – (considering the Ordering Provider)

This, but not That • Measure true response

– Filtered out patients we knew were septic before the alert • Hold alerts for ECMO patients for 24 hours

– Filter based on reliable attributes • Hold notification of SIRS for 48 hours after cardiac surgery

– Policy written by one of our hospitals • Using an OB specific screening tool (MEWT)

– Turned off alerts for OB medical services

STEPS • T= Treatment/Clinical: Our program is aimed at initiating treatment as

soon as possible when a patient is identified as having SIRS or sepsis. We notify the primary nurse when they are signed into our EHR, independent of who's chart they are looking at in that moment.

• E= Electronic Secure Data: An enterprise analytics hub that seamlessly integrates the use of clinical, revenue, cost, pharmaceutical, quality and patient behavioral data across the full care continuum is a foundational element to achieving our corporate goals. Our sepsis bio-surveillance program is a step toward that end.

• S= Savings: We are working toward an estimated $10M savings for the sepsis DRG for our enterprise.

Outcomes • Scale • Higher nurse communication rate correlates with lower mortality rate • Fewer patients progressing to severe sepsis

Unexpected outcomes • Coding opportunity project • Sepsis bundle compliance project

Recommendations • Governance and goal ownership • Focus on KPI’s

– Pick 3-4, meaningful • Optimize from the nurses’ and providers’ perspective

– Focus the alert – Leverage the workflow that is

• Provide feedback in an actionable format – Most folks will reformat your data, help them

Thank you! • alyson.dandrea@dignityhealth.org • ken.ferrell@dignityhealth.org

Appendix A - Sepsis Alert Triggers Organ Dysfunction Criteria

Facility Lactate Hypotension Bilirubin Creatinine Facility A 28.23% 59.33% 19.14% 4.78% Facility B 32.09% 53.58% 15.47% 5.30% Facility C 25.30% 60.98% 14.02% 5.49% Facility D 21.43% 59.71% 16.00% 7.71% Facility E 45.24% 35.45% 16.14% 7.20% Facility F 20.90% 65.30% 12.69% 6.34% Facility G 39.00% 47.67% 16.00% 6.33% Facility H 34.24% 46.79% 19.12% 7.99% Facility I 31.21% 50.34% 17.11% 8.39%

A-I Average: 30.63% 52.87% 16.34% 6.78%

Appendix B - SIRS Alert Triggers

SIRS Criteria

Facility High Temp Low Temp HR RR Glucose High WBC Low WBC Bands

Facility A 16.98% 5.19% 94.34% 70.75% 34.43% 71.23% 5.19% 4.25%

Facility B 21.09% 3.51% 93.50% 60.98% 38.49% 71.53% 6.15% 8.08%

Facility C 16.97% 7.75% 89.30% 67.53% 37.27% 73.43% 7.38% 2.58%

Facility D 20.07% 5.99% 89.96% 72.36% 32.57% 76.94% 4.23% 0.70%

Facility E 27.82% 10.80% 94.25% 62.53% 29.66% 67.82% 8.97% 1.38%

Facility F 20.11% 32.98% 80.70% 67.29% 29.49% 68.90% 1.34% 1.07%

Facility G 15.29% 6.42% 89.91% 69.72% 38.53% 77.06% 3.67% 1.22%

Facility H 21.59% 6.66% 92.69% 67.69% 30.52% 73.86% 5.52% 4.22%

Facility I 23.46% 10.38% 90.38% 63.46% 41.15% 64.62% 7.69% 1.15%

Average for A-I: 20.82% 9.50% 90.77% 66.76% 34.10% 72.18% 5.51% 3.00%