Informing for ImprovementReport cards, performance measures and quality indicators – why bother?

Richard HamblinCenter for Health Studies

Group Health Cooperative of Puget Sound

Why publish report cards?

• How do we encourage consumerism? Well, one thing you do is you make sure people understand their options…– G.W Bush, Minneapolis, August 22, 2006

• Our choice information pages …help you make a choice that best suits you.”– UK Department of Health choice website

But…

• “when this information is published only a minority are aware of it; of those, most do not understand it, trust it or use it…” – Gwyn Bevan 2005

My objectives

• Understand this contradiction

• Group health provides a unique population– Chronic conditions– My Group Health

• What is the likely use of such information?

• What does this imply about presentation and dissemination?

More enthusiastic users?}

How might it work

• Pressure on providers

• Patients as consumers – choosing the best providers

• Informed and empowered patients – getting the best from their providers

• Which of these two is most likely to work?

Survey

• Written survey tool mailed to 600 respondents

My GH non-userno chronic condition n = 150

My GH userno chronic condition n = 150

My GH userdiabetesn = 150

My GH non-userdiabetesn = 150

My GH non-usern = 300

My GH usern = 300

Diabetesn = 300

no chronic condition n = 300

Survey

• 22 written questions to test pre-determined hypotheses– Patients with long-term conditions are more interested

– My Group Health users are more interested

– Less satisfied patients are more interested

– Information more likely to be used for boosting confidence than changing provider

• Data collection February to April 2007

Responses

• Ethnicity similar across groups

• MyGH users wealthier and better educated

• Diabetics generally lower income

0%

10%

20%

30%

40%

50%

To

tal

MyG

H

NM

yGH

Dia

bet

es

No

CC

Response rates

Lots of interest

• 11 point scale (0-10) used to report interest

Mean Median

Weighted total 7.7 8

My Group Health User 7.9 8

My Group Health Non User

7.4 9

Diabetes 7.4 9

No chronic condition 7.9 9

But little prior knowledge or use

No meaningful differences between the groups

0%5%

10%15%20%25%30%

To

tal

MyG

H

NM

yGH

Dia

bet

es

NL

TC

Percentage respondents who knew of various report cards HEDIS

US News

Health Grades

Use of any

What do I do with this then?the continuum of potential use

Active and immediateNone

None Passive/uncertain Prospective

Active Consumerist

Active informed patient

“I would not use”

“Understand better how my doctor rates”

“Choose doctor when entering health plan”

“Boost confidence to discuss things I don’t understand or agree with.”

“Change doctor inside current plan”

Proportion of respondents citing different uses for data (forced choice of

one use)

0%

20%

40%

60%

80%

100%

Total

MyG

H

NMyG

H

Diabet

es

No CC

would not use

understand better

choose doctor (newplan)

change doctor(existing plan)

boost confidence

For all groups “understand better” is a significantly greater proportion than any other

Use versus self-reported interest does not vary (except for the would not use group)

Active users only

• Surprising result – expected a greater proportion in the active section

• Active uses only – significant results (p<0.05)

• “Boost confidence” a more common response, but very small numbers

922Diabetes*821MyGH*

Change doctorBoost confidencen

Key result

• Just because people are interested in the information doesn't mean that they are going to use it to make choices tomorrow

How do patients want data presented?

• Individual measures vs an overall service rating• Individual physicians vs hospital/practice• Benchmarks/expected performance vs rankings

• * (p<0.01)

76.1%*57.8%45.9%Total

BenchmarksIndividual physicians

Individual measures

% of responders

How does satisfaction affect interest?

• Are satisfied patients less interested in having information about quality?

• Test 1: Correlation of interest scale with CAHPS satisfaction scale

0.001No CC0.001Diabetes0.011NMyGH0.002MyGH0.000Total

r2

Correlation between interest and satisfaction ratings

How does satisfaction affect interest?

• Test 2: Comparison of interest scale with specific CAHPS attributes of patient-focused care

Mean interest scores by regularity of CAHPS attributes

7.97.5Time

7.57.7Respects

7.77.6Listens

7.77.7Explains

Not alwaysAlways

CONCLUSIONS

• Many prior expectations were wrong

• Interest in performance information uniformly high

• Prior knowledge and use of report cards uniformly low

• Contrary to expectations, little difference between groups

• No relationship between satisfaction with doctor and desire for information about quality

CONCLUSIONS

• The importance of “better understand”

• Doesn't have to be “used” to be “useful”

• Possible interpretations– Information as a resource

• Reassurance• Accountability

– Understanding as a precursor of action

CONCLUSIONS

• Large majority in all groups favoured comparisons with benchmarks rather than ranked performance

• Consistent with the “use” finding

Policy implications – consumerisms’ weaknesses and an alternative approach

• Publishing information about quality will not necessarily encourage choice

• Not because the data are badly presented but because most patients don’t prioritise choice

• A different goal of trust and understanding of quality of service

Policy implications – unresolved next steps

• Balancing measurement of different things: – Clinical process – outcome – experience

• How to set external benchmarks? (e.g. NQF process)

• How to determine the “normal range”? (e.g. outliers, composites)