1

Presentation for the Stockholm County Council

Noel Eldridge, MSDepartment of Veterans Affairs

National Center for Patient Safety(202) 461-6517 & noel.eldridge@va.gov

11/1/07

2

VA’s Current Patient Safety Performance Measure

Radiology Reportsverified within 2 days

• Starting point in 2005 was 67%

• Most recent quarter was 90%

• Reports in first 3 Qtrs of FY 2007: 5,255,967

• The difference between 67% and 90% is over 1,600,000 reports per year, or ~4,400 per day everyday.

3

Why this is important…

• RCA counts for 3 years shown

• Delay in Treatment/ Diagnosis/Surgery is tied with Patient Falls for most reported coded category of Root Cause Analysis submitted by VAMCs

Delay in Treatment/Diagnosis/Surgery

4

Using Administrative Data to Measure Patient Safety

• This would be great if it worked (in particular the AHRQ Patient Safety Indicators [PSIs]).

• At least three major problems:1. False Negatives (a new version of the classic

under-reporting problem plaguing patient safety)2. False Positives (the new problem based on “present

on admission” and dependence on coding)3. How to analyze data:

– designed for another purpose, – characterized by rare events, and – is often “non-Gaussian”, i.e., not “normally”

distributed

5

Under-reporting?adverse events, complications, or illnesses?

CDC’s Official Number of Deathsfor Leading Causes of Death http://webapp.cdc.gov/sasweb/ncipc/leadcaus10.html

• Heart disease: 652,486• Cancer: 553,888• Stroke (cerebrovascular

diseases): 150,074• Chronic lower respiratory

diseases: 121,987• Accidents (unintentional injuries):

112,012• Diabetes: 73,138• Alzheimer's disease: 65,965• Influenza/Pneumonia: 59,664• Nephritis, nephrotic syndrome,

and nephrosis: 42,480• Septicemia: 33,373

CDC Analysis published in Public Health Reports, March - April 2007, pp 160-166

The estimated deaths associatedwith healthcare-associated infections[for 2002] in U.S. hospitals were 98,987: • 35,967 were for pneumonia, • 30,665 for bloodstream infections, • 13,088 for urinary tract infections, • 8,205 for surgical site infections, and • 11,062 for infections of other sites.

These are from completely separate analyses…

6

False Negatives?

• 2002 VA data, from Rosen, et al, 2006, cases found: – Complications of

Anesthesia = 55– “Infection resulting from

Medical Care” (highly restricted definition) = 816

– Postoperative Sepsis = 106

• VA performs over 350,000 operations per year… compare to PSI denominators

7

False Positives?

- Decubitus Ulcer = 3316- AHRQ analysis of data from

California and NY indicates that >80% were present on admission

- Post-op Hip Fracture = 29- AHRQ data indicates >70%

present on admission - A false positive and a false

negative problem? (29 total? Leaves 10 in-hospital?)

- Mayo Clinic study indicated that overall, only 63% occurred in hospital…

8

Recent Publication (not from AHRQ -- AHRQ paper in review)

Impact of Diagnosis-Timing Indicators on Measures of Safety, Comorbidity, and Case Mix Groupings From Administrative Data Sources.

Medical Care. 45(8):781-788, August 2007.Naessens, James M. ScD, MPH *; Campbell, Claudia R. PhD +; Berg, Bjorn BA *; Williams, Arthur R. PhD *; Culbertson, Richard PhD +

Abstract: Context: Many attempts to identify hospital complications rely on secondary diagnoses from billing data. To be

meaningful, diagnosis codes must distinguish between diagnoses after admission and those existing before admission.

Objective: To assess the influence of diagnoses at admission on patient safety, comorbidity, severity measures, and case mix groupings for Medicare reimbursement.

Design: Cross-sectional association of various diagnosis-based clinical and performance measures with and without diagnosis present on admission.

Setting: Hospital discharges from Mayo Clinic Rochester hospitals in 2005 (N = 60,599). Patients: All hospital inpatients including surgical, medical, pediatric, maternity, psychiatric, and rehabilitation patients.

About 33% of patients traveled more than 120 miles for care. Main Outcome Measures: Hospital patient safety indicators, comorbidity, severity, and case mix measures with and

without diagnoses present at admission. Results: Over 90% of all diagnoses were present at admission whereas 27.1% of all inpatients had a secondary

diagnosis coded in-hospital. About one-third of discharges with a safety indicator were flagged because of a diagnosis already present at admission, more likely among referral patients. In contrast, 87% of postoperative hemorrhage, 22% of postoperative hip fractures, and 54% of foreign bodies left in wounds were coded as in-hospital conditions. Severity changes during hospitalization were observed in less than 8% of discharges. Slightly over 3% of discharges were assigned to higher weight diagnosis-related groups based on an in-hospital complication.

Conclusions: In general, many patient safety indicators do not reliably identify adverse hospital events, especially when applied to academic referral centers. Except as noted, conditions recorded after admission have minimal impact on comorbidity and severity measures or on Medicare reimbursement.

(C) 2007 Lippincott Williams & Wilkins, Inc.

9

n= 1800+

Mean = 1.0

Std. Dev. = 1.8

Median = 0

Non-normally Distributed “Rare” Events

3+ years of data for nursing home sections of VA Medical Centers – statistical analysis based on a normal distribution is wrong and/or useless here: O/E ratios, p values, etc.

10

Triangle Shirtwaist Factory Fire of 1911

• March 16, 1911 A report on fire traps is published.  The report argues that many New York City buildings lack “even the most indispensable precautions necessary.”

• March 25, 1911 Shortly before quitting time, a fire breaks out on the eighth floor of the building housing the Triangle Shirtwaist factory.  The fire kills 146 victims.

• April 2, 1911 A meeting is held to discuss concerns over lack of safe working conditions in New York City's factories.  Resolutions are passed demanding new legislation.

11

What a Triangle Shirtwaist Factory Fire Every Day Would be in U.S.

Healthcare • 146 x 365 = 53,290 deaths• 53,290 / 6,007 = 8.9 per year per hospital• 365 / 8.9 = 41 days between deaths

or• 53,290 / 36,600,000 = 0.15%

(1 in 675 admissions)• Not so hard to believe… but would be 2.1% of all

deaths in US (53,290 / 2,443,387)• IOM said 44,000 to 98,000 in 1999. Estimates since

have ranged from about 15,000 to over 200,000.