Overview of the Hospital Safety Score March 24, 2015

Missy Danforth, Senior Director of Hospital Ratings, The Leapfrog Group

Presentation Overview • Who is getting a Hospital Safety Score? • Changes to the Scoring Methodology Since October 2014 • Details of the data review process • Important Dates • Questions

2

What is the Hospital Safety Score?

• The Hospital Safety Score is an A, B, C, D, or F letter grade reflecting how safe hospitals are for patients.

• The Hospital Safety Score launched in June 2012. This fall will be the seventh release.

• More information at www.HospitalSafetyScore.org

3

What Hospitals Will Receive a Hospital Safety Score? • General, acute-care hospitals for which there is adequate

public data • Excluded Hospitals include:

• Specialty Hospitals (i.e. surgical centers, cancer hospitals, women’s hospitals, etc.)

• Critical Access Hospitals • Free –Standing Pediatric Facilities • Non-IPPS participating hospitals (hospitals from the state of MD)

• Hospitals Missing Too Much Data:

• More than 9 process measures • More than 4 outcome measures 4

SCORING OVERVIEW 5

Measure Selection Criteria • Measures are publicly-reported from national data sources,

reflecting individual hospital results • Leapfrog Hospital Survey • Centers for Medicare and Medicaid Services data sets

• Measures are endorsed or in use by a national measurement

entity

• Measures are linked to patient safety (“freedom from harm”) • Directly quantifying patient safety events • Assessing processes that lead to better outcomes • Identified by experts as important to patient safety 6

Measures included in the Hospital Safety Score Measure Name Primary Data Source Secondary Data Source

Process and Structural Measures (15) Computerized Physician Order Entry (CPOE) 2014 Leapfrog Hospital Survey* 2013 HIT Supplement I

ICU Physician Staffing (IPS) 2014 Leapfrog Hospital Survey* 2013 AHA Annual Survey I* Safe Practice 1: Leadership Structures and Systems 2014 Leapfrog Hospital Survey* Safe Practice 2: Culture Measurement, Feedback and Intervention 2014 Leapfrog Hospital Survey* Safe Practice 3: Teamwork Training and Skill Building 2014 Leapfrog Hospital Survey* Safe Practice 4: Identification and Mitigation of Risks and Hazards 2014 Leapfrog Hospital Survey* Safe Practice 9: Nursing Workforce 2014 Leapfrog Hospital Survey* Safe Practice 17: Medication Reconciliation 2014 Leapfrog Hospital Survey* Safe Practice 19: Hand Hygiene 2014 Leapfrog Hospital Survey* Safe Practice 23: Care of the Ventilated Patient 2014 Leapfrog Hospital Survey* SCIP INF 1: Antibiotic within 1 Hour CMS Hospital Compare* SCIP INF 2: Antibiotic Selection CMS Hospital Compare* SCIP INF 3: Antibiotic Discontinued After 24 Hours CMS Hospital Compare* SCIP INF 9: Catheter Removal CMS Hospital Compare* SCIP VTE 2: VTE Prophylaxis CMS Hospital Compare*

Outcome Measures (13) Foreign Object Retained Data.cms.gov Air Embolism Data.cms.gov Pressure Ulcer – Stages 3 and 4 Data.cms.gov Falls and Trauma Data.cms.gov CLABSI 2014 Leapfrog Hospital Survey* CMS Hospital Compare* CAUTI 2014 Leapfrog Hospital Survey* CMS Hospital Compare* SSI: Colon CMS Hospital Compare* PSI 4: Death Among Surgical Inpatients CMS Hospital Compare* PSI 6: Iatrogenic Pneumothorax CMS Hospital Compare* PSI 11: Postoperative Respiratory Failure data.cms.gov PSI 12: Postoperative PE/DVT CMS Hospital Compare* PSI 14: Postoperative Wound Dehiscence CMS Hospital Compare* PSI 15: Accidental Puncture or Laceration CMS Hospital Compare*

[i] AHA Annual Survey © 2013 Health Forum, LLC *Updated Since October 2014

7

A note about reporting periods • Leapfrog Hospital Survey Measures

• The 2014 Leapfrog Hospital Survey includes two reporting periods:

• Hospitals that submit a survey before September 1, 2014 were asked to report on the 12-months ending December 31, 2013.

• Hospitals that submit a survey on or after September 1, 2014 were asked to report on the 12-months ending June 30, 2014.

• Because the data snapshot date was February 15, 2015, the reporting period for the Leapfrog Hospital Survey will be listed as 01/01/2013 – 06/30/2014.

• CMS Measures

http://www.medicare.gov/hospitalcompare/Data/Data-Updated.html

8

Delays in Receiving Updated CMS Outcome Measures • Hospital-Acquired Condition Measures:

• Foreign Object Retained After Surgery • Air Embolism • Pressure Ulcers • Falls/Trauma

• Patient Safety Indicators: • PSI 11

9

Weighting Process • Two (2) measure domains, each weighted 50%:

1. Process/structural measures 2. Outcome measures

• Three (3) criteria for weighting individual measures

• Strength of Evidence (rating of 1 or 2) • Opportunity (rating of 1, 2, 3), based on coefficient of variation • Impact (rating of 1, 2, or 3), based on no. of patients possibly

affected by the event and severity of harm to individual patients

• Weight score: [Evidence + (Opportunity x Impact)] 10

Z-Score Methodology • Standardizes data from individual measures with different

scales

• Counts how many standard deviations a hospital’s score on the measure is away from the mean

• Mean always equals 0; worse than mean = negative z-score ; better than mean = positive z-score

• Translate raw score on measure to z-score: • Process/Structural Measures = [(Hospital Score – Mean)/Standard

Deviation] • Outcome Measures = [(Mean – Hospital Score)/Standard

Deviation] 11

Overall Score • Summation of z-score for each measure × weight for each

measure 3.0 + CPOE z-score × CPOE weight + IPS z-score x IPS weight +

CLABSI z-score × CLABSI weight . . . . etc. • If measure has missing data, then weight for that measure is

re-apportioned to other measures within the same domain

• 3.0 was added to each hospital’s final score to avoid possible confusion with interpreting negative patient safety scores

12

CHANGES TO THE SCORING METHODOLOGY SINCE OCTOBER 2014

13

No changes to the measure set or the scoring methodology

14

DETAILS OF THE DATA REVIEW PROCESS

15

Secure Website • http://www.leapfroggroup.org/data-validation/validation-

login

Details on what public reports were used to obtain source data. Links to source data with instructions.

Hospitals must confirm their Medicare Provider Number and Hospital name, address, and contact information before moving on to the next page.

16

Source Data

Instructions for hospital to review their source data and a link to the help desk

For each measure, hospitals are provided with the measure name, type of measure, data source (primary or secondary), reporting period, measure score (i.e. points, rate, SIR, etc)

Hospitals are asked to confirm that each Measure Score matches their score (i.e. rate, SIR, etc) from the Data Source (i.e. Leapfrog Hospital Survey Results, CMS Hospital Compare, etc) 17

Where to Locate Source Data Hospitals That Submitted a Leapfrog Hospital Survey by December 31, 2014

• CPOE: Visit www.leapfroggroup.org/cp, find the column labeled “prevent medication errors.” Leapfrog score used to determine points towards safety score (i.e. 4 bars equals 100 points, 3 bars equals 50 points, etc).

• ICU Physician Staffing: Visit www.leapfroggroup.org/cp, find the column labeled “Appropriate ICU Staffing.” Leapfrog score used to determine points towards safety score.

• CLABSI and CAUTI Standardized Infection Ratio (SIR): Visit www.leapfroggroup.org/cp, find the column labeled “prevent ICU infections.” The SIR was used to calculate in the safety score calculation, and is located in the table at the top of the page. 18

Where to find your hospital's CLABSI SIR on Leapfrog’s results website

19

Where to find Safe Practice Points Hospitals That Submitted a Leapfrog Hospital Survey by December 31, 2014

• Log into your 2014 Leapfrog Hospital Survey

20

Where to find your hospital’s CLABSI, CAUTI, SSI SIR on CMS’ results website https://data.medicare.gov/data/hospital-compare

21

Enter MPN in the search box

22

Where to find your hospital’s CLABSI, CAUTI, SSI SIR on Hospital Compare http://www.medicare.gov/hospitalcompare/search.html

23

1. Scroll down to the HAI measures 2. Select “Show All Graphs”

24

Where to Locate Source Data CMS Data for All Hospitals

• SCIP Measures: Hospital rates are published by CMS at https://data.medicare.gov/Hospital-Compare/Timely-and-Effective-Care-Hospital/yv7e-xc69

• PSI 4, 6, 12, 14, 15: Hospital rates are published by CMS at

https://data.medicare.gov/Hospital-Compare/Readmissions-Complications-and-Deaths-Hospital/7xux-kdpw

25

Where to Locate Source Data AHA Data for Hospitals That Did Not Complete a Leapfrog Hospital Survey by August 31, 2014

• ICU Physician Staffing - 2012 AHA Annual Survey • CPOE – 2013 HIT Supplement

• Please contact AHA Health Forum for more information

26

What if the Measure Score doesn’t match the public report? • Hospitals are asked to contact

the help desk immediately once they have confirmed the measure and reporting period.

• Hospitals must provide a copy

of the public report that shows a different score

• If we find a recording error, we will update the score and re-issue a numerical safety score

27

Preview Numerical Safety Score

28

Important Dates • February 15th – Data Snapshot Date • March 16th - Letter sent to CEOs of hospitals receiving a Hospital Safety Score.

Letter included: • Information about the Hospital Safety Score • A username/password to a secure website where hospitals can review the source

data that Leapfrog used to calculate their numerical safety score • Links to the Hospital Safety Score help desk and helpful documents

• March 18th to April 8th - Courtesy Data Preview Period

• TBA – Hospitals will be able to preview letter grades 48 hours prior to the pubic

release (www.HospitalSafetyScore.org). • TBA – Letter Grades will be published at www.HospitalSafetyScore.org.

• For more information about important dates, visit:

http://www.hospitalsafetyscore.org/for-hospitals/updates-and-timelines-for-hospitals

29

More Information • Hospital Safety Score Help Desk -

scorehelp@leapfroggroup.org

• Hospital Safety Score Website – www.HospitalSafetyScore.org

30