Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures

Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures

Elizabeth A. McGlynn, Ph.D.Associate Director, RAND Health

September 28, 2007

McGlynn AHRQ Efficiency 2 09/28/07

The Work Was Done by A Multidisciplinary Team

• Peter Hussey• John Romley• Han de Vries• Margaret Wang• Paul Shekelle• Dana Goldman• Martha Timmer• Susan Chen• Jason Carter• Carlo Tringale


Overview of Talk

• Highlight motivation for current work

• Present RAND’s typology

• Review existing measures

• Examples

• Concluding thoughts


Many Fortune 50 Companies Are Demanding Cost and Quality Metrics on Physicians

High/Low High/High

Low/Low Low/High

Efficient

Effective


Purchasers Are Using Efficiency Metrics In Several Ways

• Public reporting – information to help consumers make more cost-conscious decisions

• Pay-for-performance – financial rewards to providers with better performance

• Tiering – differential co-payments to encourage patients to choose higher performing providers

• Selective contracting – contracts limited to providers who perform at a certain level


But…• Little is known about what various stakeholders

mean by “efficiency”– Considerable lack of common language,

conceptual clarity– Link between content of metrics and proposed

applications not always thought through

• Little is known about the consequences (intended and unintended) of applying available metrics at different levels in the system

– Can measures developed for one level (health plan) be applied at a different level (physician)?


Overview of Talk




• Examples



Definition of Efficiency Measure

The relationship between a specific product of the health care system (output) and the resources used to create that product (inputs)

• Maximize output produced for a given input• Minimize inputs used for a given output


Efficiency Measures Typology Overview

1. Perspective

2. Output(s)

3. Inputs

Who is asking what about whom, and why?

What is being produced?

What resources are used to produce the output?

Typology is organized in three tiers


Perspective

• We identify several potential points of view:– Health care “firms”

• Physicians• Hospitals• Nursing homes

– “Agents”• Health plans• Employers

– Individuals– Society


Context Matters in Efficiency Measurement

Stakeholder

Who Are TheyResponsible For?

Over What Time Period?

Physician Active patients in a panelAs long as responsible

for patient

Hospital Patients who are admitted

During hospital stay

Health Plan Enrollees Renewed annually

Employer Employees and covered dependents

Length of employment

Society All residents Unlimited


Use of Information Varies With Perspective

• Health care firms can change the way the outputs are produced

– Amount and/or mix of inputs

– Service amenities associated with a product

• Agents can principally change how much is paid, the conditions under which the product is purchased, or how the product is bundled

• Individuals can principally change what they buy and from whom


Output: What’s Being Produced?• We define two major categories of outputs:

– Health services

– Health outcomes

• Being explicit about the output is critical (and often not done)

• Producers tend to define outputs

– Financial flows (how the product is purchased) influence definitions, for example:

• Hospital day vs. a discharge

• Quality adjusted life year vs. a covered life


Examples of Outputs by Type

Entity Service Output Examples

Health Outcome Output Examples

Physician

• Visits• Procedures• Diagnoses• Prescriptions

• Preventable deaths• Quality adjusted life

years

Hospital• Discharges• Procedures• Inpatient days

• Functional status• Preventable deaths• Preventable

complications

Health Plan • Covered lives• Quality adjusted life

years


Inputs

• We identify two main ways of measuring inputs:– Physical– Financial

• These map to the economic definitions of technical (physical) and productive (financial) efficiency


Technical and Productive Efficiency MeasuresPoint to Different Root Causes of (In)Efficiency

Technical Efficiency

Inputs are put to good use

Productive Efficiency

Inputs are put to good use

Best mix of inputs chosen

Lowest prices are paid

+

+


Social Efficiency

• Social efficiency is achieved when no member of society can be made better off without making another member worse off

– Giving more resources to one person implies that those resources have been taken away from someone else

– Appeal of “waste” is the notion that those resources do not benefit anyone currently


RAND’s Efficiency Typology

Society

Providers PurchasersHealth Plans

Health Care Firms

Individuals

Health Services Health Outcomes

Physical Financial

Per

spec

tive

Out

put

Inpu

t


Overview of Talk




• Examples



Identifying Existing Efficiency Measures

• Peer-reviewed literature– Medline and EconLit search: 1990-2005– Titles, abstracts, and articles reviewed by 2

independent reviewers; consensus resolution– Excluded non-U.S. studies

• “Gray” literature review– Purposive, reputational sampling– 8 vendors identified & reviewed


Overview of Article Flow

Titles (n=4,324)

Rejected (n=3,754)

Articles (n=563)

Excluded (n=145)

Detailed review (n=318)

High priority (n=158)


Measures in Literature Dominated by Hospital Perspective & Service Outputs

0 20 40 60 80 100 120

Physician

Hospital

Number of measures

Hosp disch/days Procedures MD visits

Other services Episodes Outcomes


40 Different Physical Inputs Identified in Peer Reviewed Literature

• Physician labor – number of physicians (usually FTEs) or hours worked

• Nursing labor – number of nurses (usually FTEs) or hours worked

• Administrative, technical, or other labor categories – number of personnel (usually FTEs) or hours worked

• Beds – the most common indicator of capital stock

• Depreciation of assets - a measure of capital, calculated in various ways

Financial inputs appear almost as often as physical inputs


No measures using health services as outputs explicitly incorporated a

measurement of quality

Methods for incorporating or accounting for quality are not well

developed in peer reviewed literature


Different Worlds of Efficiency Measures

• There is an almost total separation between the published studies of health care efficiency and the use of efficiency measures by providers, payers, and purchasers

– Academic measures focus on multiple input/output models

– Vendor measures focus on specifying outputs

• Purchasers and health plans are generally using measures developed by vendors

– Little independent testing of reliability and validity has been reported on these metrics


Vendor-Developed Measures

• Episode-based: ETGs, MEGs, CCGs

– Claims aggregated into clinically-defined episodes and attributed to different entities

– Measure is cost per episode (productive efficiency)

– Also can look at resource use per episode (technical efficiency)


Vendor-Developed Measures

• Population-based: ACGs, DxCGs, CRGs, RRU, PPMS

– Patient populations weighted by morbidity burden

– Measure is cost per risk-adjusted patient per year (productive efficiency)

– Also can look at resource use (technical efficiency)


Overview of Talk




• Examples


Some Examples of Efficiency MeasuresMetric Perspective Outputs Inputs

Cost per episode Health plan Health service bundle (cross entity)

Monetized total cost

Cost per discharge

Health plan Health service bundle (w/in entity)

Monetized total cost

Cost per covered life

Employer # employees with health insurance

Premium price charged by plan

Cost per health improvement

Medicare Change in functional status

Total costs of care

Labor utilization Hospital (internal)

Total number of discharges

Total number of nursing hours

Productivity Physician (internal)

Number of patients seen per year

Total number of MD pt-care hours

Generic prescribing rate

Health plan Number of days rx supplied

Number of days generic rx supplied


Measures We Wouldn’t Classify as Efficiency

Metric Perspective Output Input Why Not?

Readmissions Medicare Discharges Number of admissions within 30 days of discharge

Missing key events (death, different facility)

Rate of CABG surgery

Employer Total number of enrollees

Number of bypass surgeries

No input specified

Cost-effectiveness

Society Change in outcome

Change in cost (from using new vs. existing intervention)

Result not specific to an entity; may help construct metrics


We Asked What the Evidence Was Regarding Whether the Measure Was…

• Important• Scientifically sound• Feasible• Actionable


Summary of Findings

Criterion Peer Reviewed Vendor

Important Low High

Scientifically sound Low Low

Feasible Low High

Actionable Low Low


Overview of Talk




• Examples



Conclusion

• Disconnect between academic world and vendors on efficiency measurement offers opportunity

• Although steps have been taken to achieve consensus among stakeholders, more work needs to be done

– Typology can assist in structuring more explicit discussions about key issues

• Scientific soundness of both academic and vendor-developed measures is largely unknown

• Ability of potential users to act on the results of these measures is not well understood


Some Priorities for the Future• Be explicit about the perspective, output, and

inputs used in measure:

– What’s the purpose, approach?

• Develop measures to fill important gaps

– When is it most important to incorporate quality measures?

– Measures that use health outcomes as outputs

– Measures for units of analysis other than hospital

• Evaluate whether efficiency measures are scientifically sound, feasible, actionable

Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures

Documents

Transcript of Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures