Do Good Nursing Homes Achieve Good Outcomes? Measurement Considerations Associated with Public...
-
Upload
godwin-chambers -
Category
Documents
-
view
217 -
download
0
Transcript of Do Good Nursing Homes Achieve Good Outcomes? Measurement Considerations Associated with Public...
Do Good Nursing Homes Achieve Good Outcomes? Measurement Considerations Associated with Public Reporting
Vincent Mor, Ph.D. Brown University
Do Good Nursing Homes Achieve Good Outcomes?
2
Purpose
Review Long Term Care Assessments Using Nursing Home as Example, Review Long Term
Care Outcome Domains and Measures Present Empirical Data Addressing Select Nursing
Home Quality Measurement Issues How Correlated are Quality Measures? How Stable are Quality Measures? How Variable are Quality Measures?
Implications for Public Reporting
Do Good Nursing Homes Achieve Good Outcomes?
3
Background Long Term Care, while relatively deficient in
resources, had head start with uniform clinical assessment tools in Rehab Hospitals, nursing homes and home health agencies
FIM for Rehab Hospitals, MDS for nursing homes and OASIS for home health emerged from acknowledged need to improve care quality.
MDS and OASIS implemented and applied to policies ranging from case mix reimbursement to quality review
Do Good Nursing Homes Achieve Good Outcomes?
The Nursing Home Resident Assessment Instrument (RAI)
1986 Institute of Medicine Report on Nursing Home Quality Recommended a Uniform Resident Assessment Instrument to Guide Care Planning
OBRA ‘87 Contained Nursing Home Reform Act Including RAI Requirement
A 300 Item, Multi-Dimensional RAI – MDS was Tested for 2 Years
Mandated Implementation in 1991 Fully Computerized in 1998 Case Mix Reimbursement in 1999 Quality Reporting in 2002
Do Good Nursing Homes Achieve Good Outcomes?
5
The Outcome and Assessment Information Set (OASIS)
Initially designed as an outcome measurement tool for home health agency staff to track how well patients progressing
Transformed into a multi-purpose tool for: patient assessment and care planning for individual adult
patients agency-level case mix reports internal HHA performance improvement Case-Mix Reimbursement mandated by BBA National Home Health Quality Compare 2003
Do Good Nursing Homes Achieve Good Outcomes?
6
Functional Impact Measure (FIM) Initiated as a way to document functional
progress during rehabilitation Widely used in Rehab Hospitals around the
country Measures Mobility, ADL dependence and
Cognition Summary scores created and differences
between admission & discharge calculated Some provider performance profiling
underway
Do Good Nursing Homes Achieve Good Outcomes?
7
Nursing Home Measurement Issues
What does quality mean and to whom? Hard to know how much inter-state variation
is due to measurement vs. real processes Differentiating between providers; how much
of a difference is important? Inter-relationship among the domains and
measures; implications of composite scores Stability of measures over time
Do Good Nursing Homes Achieve Good Outcomes?
8
Domains of Nursing Home Quality Measures Regulatory Compliance
Deficiencies, Complaints, etc. Staffing
Hours/Resident day, Skill Mix, contractors, etc. Clinical Processes
Restraints, Psychotropics, etc. Clinical Outcomes
ADL decline, Unmet needs, hospitalization Quality of Life
Mood, Food, Activity, Social Interaction Satisfaction
Service meets Expectations? Getting what you want? etc.
Do Good Nursing Homes Achieve Good Outcomes?
9
Regulatory Based Quality Measures State surveyors conduct annual inspections
following national protocol to assess compliance with Conditions of Participation
Substantial inter & intra-state variation in number, type, severity and distribution of regulatory infractions
Factors associated with deficiencies also differ somewhat by state
Termination of Medicare/Medicaid due to regulatory violations rare
Do Good Nursing Homes Achieve Good Outcomes?
10
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
10.8 - 19.4
8.9 - 10.2
7.0 - 8.6
4.7 - 6.5
State Average Total Number of Deficiencies Per Nursing Home, 2004
Source: OSCAR 2004.
Do Good Nursing Homes Achieve Good Outcomes?
11
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
13.3% - 35%
10.1% - 13.2%
7.7% - 10.1%
4.7% - 7.5%
Percentage of NHs Cited for Causing Actual Harm orImmediate Jeopardy to Redisents in Each State, 2004
Source: OSCAR 2004.
Do Good Nursing Homes Achieve Good Outcomes?
12
02
04
06
0
%
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Source: OSCAR.
Trends and Inter-State Variations in the Percentage of FacilitiesCited for Actual Harm or Immediate Jeopardy to Residents, 1995-2004
Do Good Nursing Homes Achieve Good Outcomes?
13
Staffing as Structural Quality Measures
Historically, most measures of nursing home quality revolved around staffing levels
The mix of staffing has also been considered, particularly the ratio of RNs to practical nurses and to nursing assistants
The availability of Medical Directors and Nurse Practitioners is a more recent focus
Data Quality poor since reported by home; we apply longitudinal cleaning algorithms
Do Good Nursing Homes Achieve Good Outcomes?
14
Trends in Total Staffing Levels
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1996 1997 1998 1999 2000 2001 2002 2003 2004
Hartford Panel(4.44+ HPRD)
CMS Optimum(3.90+ HPRD)
CMS Preferred(3.00+ HPRD)
CMS Minimum(2.75+ HPRD)
Below Minimum(< 2.75 HPRD)
Do Good Nursing Homes Achieve Good Outcomes?
15
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
78.7 - 96.5
66.8 - 78.6
56.8 - 66.7
32.3 - 56.7
22.4 - 32.2
Percent Nursing Homes Exceeding CMS PreferredTotal Direct Care Staffing Level (3.0+ HPRD)
Source: OSCAR 2004.
Do Good Nursing Homes Achieve Good Outcomes?
16
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
11.7 - 30.0
8.5 - 11.6
5.1 - 8.4
3.6 - 5.0
0.0 - 3.5
Percent Nursing Homes Exceeding Hartford RecommendedTotal Direct Care Staffing Level (4.44+ HPRD)
Source: OSCAR 2004.
Do Good Nursing Homes Achieve Good Outcomes?
17
Proportion of facilities using 5% or more
contract RNs or LPNs, 1992-2001
0
2
4
6
8
10
12
14
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
%
RN LPN
Do Good Nursing Homes Achieve Good Outcomes?
18Note: Difference between unadjusted & adjusted means is statistically significant (at p<.001) by paired T-test for all years 2000-2004.
Mean Staffing HPPD, 1999-2004
2.88
2.91
2.95
3.043.02
3.10
2.88 2.88
2.90
2.96
2.93
2.97
2.75
2.80
2.85
2.90
2.95
3.00
3.05
3.10
3.15
1999 2000 2001 2002 2003 2004
HP
PD
Unadjusted RUG-NCMI Adjusted
Do Good Nursing Homes Achieve Good Outcomes?
19
MDS Resident Assessment Data Quality Issues
Although inter-rater reliability studies show good results, some studies suggest data MDS not consistently collected across providers.
Inconsistency of data collection undermines comparability of quality measures
Analyses of largest reliability trial done reveals large inter-state differences in the “direction” of disagreements; this is related to inter-state variation in quality indicators
Do Good Nursing Homes Achieve Good Outcomes?
20
Process Measures of Quality
Calculating the rate at which treatments that should be delivered to certain patients are and the rate at which certain things are done that shouldn’t be done Restraints Anti-psychotics Advanced Directives in Cognitively Impaired Feeding Tubes in the Cognitively Impaired
Do Good Nursing Homes Achieve Good Outcomes?
21
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
0.070 - 0.141
0.050 - 0.064
0.036 - 0.047
0.015 - 0.035
State Averaged Quality Measure, 2004:Prevalence of Restraint Use
Do Good Nursing Homes Achieve Good Outcomes?
22
Changing Rate of restraint use in facilities 1999-2004
0.1
.2.3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Restraints
Do Good Nursing Homes Achieve Good Outcomes?
23
Outcome Measures of Quality Clinical measures related to “sentinel”
health events; rare but should be near “0” Functional decline rates calculated as
changes between assessments Incidence of clinical events that are not
desired but hard to prevent - hospitalization “Unmet” need for care Measures of Satisfaction from family or
resident
Do Good Nursing Homes Achieve Good Outcomes?
24
TX
CA
MT
AZ
ID
NV
NM
CO
IL
OR
UT
KS
WY
IA
SD
NE
MN
ND
FL
OK
WI
WA
MO
AL GA
AR
LA
MI
IN
PA
NY
NC
MS
TN
VA
KY
OH
SC
ME
WV
MI VT NH
NJ
MA
CT
MD DE
RI
0.177 - 0.244
0.154 - 0.177
0.140 - 0.153
0.126 - 0.140
State Averaged Quality Measure, 2004:Incidence of Late-loss ADL Worsening
Double the rate between lowest & highest state.
Do Good Nursing Homes Achieve Good Outcomes?
25
Hospitalization Rate within 6 Months in the Long-Stay NH Population per State in 2000
0
5
10
15
20
25
30
LA
MS
NJ
OK
TX
KY
AR
WV
GA IL FL
MO AL
TN
MD
OH
PA IN MI
NY IA SD
CA
VA
MA
NC
SC RI
NV
DE
KS
NE
MN
WY
CT
WI
AZ
CO
WA
ND
MT
VT ID OR
NH
ME
NM
UT
% H
ospitaliz
ed
Source: MDS 2000; Medicare inpatient claims 2000.
Do Good Nursing Homes Achieve Good Outcomes?
26
How have clinical quality measures changed over the last half decade?
Pressure Ulcer Prevalence remained stable Anti-psychotic medication use increased Restraint use declined
HOWEVER Lots of variation; Facilities in the Top Quintile in 1999 converged to
the middle; and vice versa Restraint Improvement resulted from reductions
in restraint use among highest users
Do Good Nursing Homes Achieve Good Outcomes?
27
Changing Rate of ADL decline in nursing homes 1999-20040
.1.2
.3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
ADL Decline
Do Good Nursing Homes Achieve Good Outcomes?
28
Change in the Rate of Restraint Use: Stratified by Quintile Ranking at Baseline (1999Q3)
0.000
0.050
0.100
0.150
0.200
0.250
0.300
19
99
Q3
19
99
Q4
20
00
Q1
20
00
Q2
20
00
Q3
20
00
Q4
20
01
Q1
20
01
Q2
20
01
Q3
20
01
Q4
20
02
Q1
20
02
Q2
20
02
Q3
20
02
Q4
20
03
Q1
20
03
Q2
20
03
Q3
20
03
Q4
20
04
Q1
20
04
Q2
20
04
Q3
20
04
Q4
Me
an
Ra
te
QUINTILE 5
QUINTILE 4
QUINTILE 3
QUINTILE 2
QUINTILE 1
Do Good Nursing Homes Achieve Good Outcomes?
29
Do Nursing Homes that start out with good clinical outcomes remain good?
Split all US nursing homes into quintiles in 1999 based upon their clinical quality
Tracked facilities’ clinical quality measures each quarter over next 5 years
Observed substantial “regression to the mean”
Most homes beginning in the top quintile ended up very similar to those beginning at the bottom
Do Good Nursing Homes Achieve Good Outcomes?
30
Change in the Rate of ADL Decline: Stratified by Quintile Ranking at Baseline (1999Q3)
0.000
0.050
0.100
0.150
0.200
0.250
1999
Q3
1999
Q4
2000
Q1
2000
Q2
2000
Q3
2000
Q4
2001
Q1
2001
Q2
2001
Q3
2001
Q4
2002
Q1
2002
Q2
2002
Q3
2002
Q4
2003
Q1
2003
Q2
2003
Q3
2003
Q4
2004
Q1
2004
Q2
2004
Q3
2004
Q4
Mea
n R
ate
QUINTILE 5
QUINTILE 4
QUINTILE 3
QUINTILE 2
QUINTILE 1
Do Good Nursing Homes Achieve Good Outcomes?
31
Measuring Differences between Providers Variation is reduced after aggregating
measures to the level of the provider Number of residents per facility means little
power in standard statistical tests Many measures are skewed Comparing averages versus rates proportion
meeting a threshold; basis for setting thresholds?
Are we seeing differences when there are none?
Do Good Nursing Homes Achieve Good Outcomes?
32
Facility ADL Decline Rates: 1999-20050
24
68
Density
0 .2 .4 .6 .8qi: adl(28) decline 4 points--adj rate
Do Good Nursing Homes Achieve Good Outcomes?
33
Facility Restraint Rates: 1999-20050
510
15
Density
0 .2 .4 .6 .8 1qi: restraints --unadj rate
Do Good Nursing Homes Achieve Good Outcomes?
34
60% of customers satisfiedIn homes in bottom 20%
Do Good Nursing Homes Achieve Good Outcomes?
35
Facility level distribution of average resident satisfaction scores: RI 2006
.5 range out of 5 points
Do Good Nursing Homes Achieve Good Outcomes?
36
Facility level distribution of average family satisfaction scores: RI 2006
.5 range out of 5 points
Do Good Nursing Homes Achieve Good Outcomes?
37
How do Quality Measures Inter-relate?
Nearly as many providers with low rates of physical restraints have high levels of pressure ulcers as have low levels;
The average correlation among MDS based measures of quality across all US nursing homes is low;
Structural and Regulatory based measures of quality are only minimally related to outcome based measures of quality
Do Good Nursing Homes Achieve Good Outcomes?
38
Average within-MSA rank-order correlation coefficients between Clinical & Reputation Quality in 2004 (weighted by # NHs in MSA):
OSCAR “Reputational” Measures MDS Clinical Measures
N(MSAs)[A] [B] [C] [D] [E] [F] G] [H]
[A] OCCUPANCY 1.000 296
[B] % PRIVATE PAY 0.126 1.000 296
[C] DEFICIENCIES -0.133 -0.138 1.000 296
[D] TOTAL NURSE HPPD -0.079 0.165 -0.125 1.000 296
[E] ADL DECLINE -0.053 0.082 0.063 0.031 1.000 296
[F] RESTRAINTS -0.021 -0.055 0.094 -0.052 -0.012 1.000 296
[G] PU WORSENING -0.120 0.045 0.097 0.025 0.213 0.045 1.000 296
[H] PERSISTENT PAIN -0.081 -0.082 0.104 -0.038 0.017 0.060 0.065 1.000 296
Do Good Nursing Homes Achieve Good Outcomes?
39
How well do the “best” homes perform?
“Best” in terms of the fewest regulatory deficiencies, most staff, etc.
Classified facilities into thirds on Deficiencies, Staffing, etc.
Then, compared them on clinical quality measures over multiple years
Do Good Nursing Homes Achieve Good Outcomes?
40
Change in Rate of ADL Decline, Stratified by Baseline (1999) Ranking of Total Deficiencies (LOW/MIXED/HIGH)
0.0000.0200.0400.0600.0800.1000.1200.1400.160
1999
Q3
1999
Q4
2000
Q1
2000
Q2
2000
Q3
2000
Q4
2001
Q1
2001
Q2
2001
Q3
2001
Q4
2002
Q1
2002
Q2
2002
Q3
2002
Q4
2003
Q1
2003
Q2
2003
Q3
2003
Q4
2004
Q1
2004
Q2
2004
Q3
2004
Q4
Mea
n R
ate
LOW MIXED HIGH
Do Good Nursing Homes Achieve Good Outcomes?
41
Change in Rate of ADL Decline, Stratified by Baseline (1999) Ranking of Total Nurse HPPD (LOW/MIXED/HIGH)
0.0000.0200.0400.0600.0800.1000.1200.1400.160
1999
Q3
1999
Q4
2000
Q1
2000
Q2
2000
Q3
2000
Q4
2001
Q1
2001
Q2
2001
Q3
2001
Q4
2002
Q1
2002
Q2
2002
Q3
2002
Q4
2003
Q1
2003
Q2
2003
Q3
2003
Q4
2004
Q1
2004
Q2
2004
Q3
2004
Q4
Mea
n R
ate
LOW MIXED HIGH
Do Good Nursing Homes Achieve Good Outcomes?
42
Does Clinical Quality Predict Nursing Home Termination?
NHs with fewest deficiencies less likely to be terminated
NHs with lowest ADL decline and restraint rate less likely to be terminated
BUT, pressure ulcer worsening or persistent pain performance not related to future termination & staffing barely predictive
Do Good Nursing Homes Achieve Good Outcomes?
43
Quality Predicting Nursing Home Closure w/in 2 years
0
0.1
0.2
0.3
0.4
0.5
0.6
Percent Closed
WorstQuality
MixedQuality
BestQuality
StaffingDeficienciesADL DeclineRestraintsVery Rare Event
Do Good Nursing Homes Achieve Good Outcomes?
44
Implications for Public Reporting and Pay for Performance
Cross-sectional Comparisons are suspect Comparisons across state lines suspect Volatility in measures within facility Measures aren’t correlated; P4P needs a
priori value based weighting Multi-dimensional quality measures mean
consumers will need support in using the data to inform decisions
Do Good Nursing Homes Achieve Good Outcomes?
45
Implications for Quality Measurement
Improving Data Quality is essential; applies to regulatory, staffing, MDS based data and patient/family surveys
Addressing Methodological issues essential before applying these data to P4P
Findings are likely applicable to Home Health Agencies and Rehab Hospitals