www.echo-health.eu

ECHO –

European Collaboration for Healthcare Optimization An international project on healthcare performance

www.echo-health.eu

The concept

• ECHO has set about the task of bringing together patient-level data from Austria, Denmark, England, Portugal, Slovenia and Spain, making them comparable.

• ECHO is expanding the usual approach in healthcare performance international comparison (built upon average values), adding variation within and across countries.

• Performance dimensions : utilization of effective care, equitable access to effective care, hospital undesirable outcomes, potentially avoidable hospitalizations, utilization of low-value procedures, technical efficiency.

• Healthcare areas or hospital providers are flagged as good- or poor performers – not a diagnostic tool but a screening tool

www.echo-health.eu

Geographicapproach Hospital-specificapproach

Researchquestion

Doestheplaceofresidence

influencethepopulationexperienceofgettinghigh-value

care?

Istheexposuretohigh-qualityand

safecaredependantontheproviderwhereapatientisassisted?

Mainendpoint

StandardisedrateorStandardisedUtilizationRatioandExcesscasesforhospitaladmissionsorprocedures

AdjustedriskandObservedtoExpectedRatio,analysingeventsamenabletohealthcarequality

Denominator

Populationlivinginapre-definedgeographicalarea

Patientstreatedinahospital

Mainaudience

Policy-makersManagersClinicians

www.echo-health.eu

MESSAGES FROM ECHO WITH IMPACT IN POLICY MAKING

www.echo-health.eu

UNEVEN UTILIZATION OF EFFECTIVE PROCEDURES

www.echo-health.eu

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

Q1 Q2 Q3 Q4 Q5

PCI and burden of ischemic disease P

opu

lation r

ate

s o

f pro

ced

ure

s

Burden of ischemic disease

Each dot represents a healthcare area – age-sex standardized rate. Q1 to Q5 represent quintiles of ischemic

disease rates. Q1 the lowest burden, Q5 the highest burden.

www.echo-health.eu

PCI utilization across welfare levels

0

6

12

18

24

30

2002 2003 2004 2005 2006 2007 2008 2009

Stan

dar

dis

ed R

ate

Q1 Q2 Q3 Q4 Q5

0

6

12

18

24

30

2002 2003 2004 2005 2006 2007 2008 2009

Stan

dar

dis

ed R

ate

Q1' Q2' Q3' Q4' Q5'

0

6

12

18

24

30

2002 2003 2004 2005 2006 2007 2008 2009

Stan

dard

ised

Rat

e

Q1 Q2 Q3 Q4 Q5

0

6

12

18

24

30

2006 2007 2008 2009

Stan

dar

dis

ed R

ate

Q1 Q2 Q3 Q4 Q5

England Denmark

Portugal Spain

Red line represents utilization in the most affluent areas; blue line represents utilizations in the worse-off

www.echo-health.eu

UNEVEN UTILIZATION OF LOW-VALUE PROCEDURES

www.echo-health.eu

DENMARK ENGLAND PORTUGAL SLOVENIA SPAIN

SR 40.13 19.55 7.02 21.45 8.58

EQ5-95 1.66 4.69 46.80 3.54 35.36

SCV 0.16 0.28 2.19 0.20 1.26

Dots represent areas – age-sex standardized rates (natural scale on the left, normalized scale on the right).

SR: standardized rate; EQ5-95: ratio between the 5th and 95th percentile. SCV: Systematic Component of Variation.

C-section in low risk deliveries

www.echo-health.eu

Excess-cases per Local Authority

Excess-cases with regard to the reference area. compares each area with those in the 10th percentile.

The darker the colour the more the number of cases that could be avoided.

Minimisation to p10

www.echo-health.eu

SYSTEMATIC VARIATION IN UNDESIRABLE EVENTS AFTER SURGERY

www.echo-health.eu

PTE & DVT after surgery

Denmark England Portugal Slovenia Spain

aIR 1,43 7,55 0,87 4,36 1,87

EQ 25-75 2,11 1,48 3,84 2,04 3,16

Dots represent hospitals - adjusted incidence of Pulmonary Thromboembolism and Deep Venous Thrombosis

(natural scale on the left, normalized scale on the right). Table: aIR: adjusted incidence rate; EQ25-75: ratio

between the 1st and 3rd quartile.

www.echo-health.eu

-50

0

50

100

150

200

Ad

juste

d C

ase

Fata

lity R

ate

x 1

,000

pa

tie

nts

0 500 1000 1500

Patient undergoing surgery

CI-95 CI-99

RISK-ADJUSTED CASE FATALITY RATES - (CABG)

Dots represent hospitals - adjusted case-fatality rate in patients undergoing CABG. Hospitals beyond the

upper confidence interval represent either alerts (beyond the red line) or alarms (beyond the dashed-line)

Conversely, those hospitals beyond the lower confidence intervals are good or the best performers.

www.echo-health.eu

Trends on undesirable events’ rates

Bubbles represent hospitals – trends in adjusted case-fatality rate in patients undergoing PCI. The bigger the

bubble the more the number of patients treated. Colours represent the position of each hospital with regard to the

benchmark and the confidence intervals – from orange the highest case-fatality rates, to dark-blue the lowest.

www.echo-health.eu

UNEVEN OUTCOMES VS. UNEVEN RESOURCES USE

www.echo-health.eu

SH

OR

TE

R L

OS

LESS MORTALITY

Hospitals are classified according to resources use (length of stay (LOS) as a proxy) and Outcomes (e.g. case

fatalities). A common benchmark has first to be estimated for both resources use and outcomes (blue diamond)

www.echo-health.eu

Each dot represents the hospital behaviour in terms of Length of Stay and Mortality rates, in patients with

Myocardial Infarction. Denmark highlighted.

www.echo-health.eu

METHODOLOGICAL ISSUES

www.echo-health.eu

• Building a homogeneous knowledge infrastructure

http://www.echo-health.eu/handbook/infrastructure.html

• Developing comparable indicators across different languages

http://www.echo-health.eu/handbook/getting-indicators.html

• Dealing with population size heterogeneity

http://www.echo-health.eu/handbook/unit_analysis.html

• Using proper analyses meant to elicit systematic and unwarranted variation

http://www.echo-health.eu/handbook/metrics.html

• Building common performance benchmarks

www.echo-health.eu

POPULATION SIZE HETEROGENEITY

www.echo-health.eu

Knee replacement

-4

-3

-2

-1

0

1

2

USA ENG POR DNK SPN

Rate 89.9 57.9 27.3 70.6 48.8

RV5-95 1.9 2.5 8.7 2.1 4.1

< 50,000 2% 80% 65% 7%

www.echo-health.eu

Former 278 units New 42 areas

www.echo-health.eu

STATISTICS ELICITING UNWARRANTED AND SYSTEMATIC VARIATION

www.echo-health.eu

0

1

2

3

4

5

6

7

8

9

10

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Extremal Quotient vs. Component of Sytematic Variation

Ex

tre

ma

l Q

uo

tie

nt

COMPONENT OF SYSTEMATIC VARIATION

www.echo-health.eu

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Em

pír

ical B

ayes

Coefficient of Variation

Coefficient of Variation vs. EB statistic

www.echo-health.eu

Variance partitioning: spatial fraction of variance

SFV: 0.55

SFV: spatially correlated /heterogeneity + spatially correlated

www.echo-health.eu

BENCHMARKING

www.echo-health.eu

National, international, aspirational benchmarking

www.echo-health.eu

Individual data vs. aggregated data: more to less sensitivity

CABG Spain : on the left individual logit-based benchmarking; on the right, aggregated

Poisson-based benchmarking

www.echo-health.eu

Pooling data vs. applying external coefficients

-50

0

50

100

150

200

Ad

juste

d C

ase

Fata

lity R

ate

x 1

,000

pa

tie

nts

0 500 1000 1500

Patient undergoing surgery

CI-95 CI-99

CABG Spain – England: on the left pooled benchmarking; on the right English estimates are

applied to Spanish data

www.echo-health.eu

more at www.echo-health.eu

www.echo-health.eu

BUILDING A HOMOGENOUS INFRASTRUCTURE

www.echo-health.eu

www.echo-health.eu

PATIENT ALLOCATION

TO THE AREA OF RESIDENCE

www.echo-health.eu

BUILDING COMPARABLE INDICATORS

www.echo-health.eu

www.echo-health.eu

www.echo-health.eu

Mapping-out codes across disease classifications

www.echo-health.eu

Variance partitioning: spatial fraction of variance

SFV: 0.55

SFV: spatially correlated /heterogeneity + spatially correlated

www.echo-health.eu

POTENTIALLY AVOIDABLE HOSPITALIZATIONS IN CHRONIC CONDITIONS

www.echo-health.eu

DENMARK ENGLAND PORTUGAL SLOVENIA SPAIN

SR 93.36 56.15 33.96 61.86 47.02

EQ5-95 1.46 1.38 2.01 2.25 2.44

SCV 1.09 0.14 0.17 0.14 0.11

Potentially avoidable hospitalizations

Dots represent areas – age-sex standardized rates (natural scale on the left, normalized scale on the right).

SR: standardized rate; EQ5-95: ratio between the 5th and 95th percentile. SCV: Systematic Component of Variation.

www.echo-health.eu

DENMARK ENGLAND

PORTUGAL SPAIN

Blue line represents the evolution of the average age-sex standardized rates Green dots represent the evolution .

of the Systematic Component of Variation.

www.echo-health.eu

Larger areas, lesser variation

Tonsillectomy C-section Prostatectomy

Larger Smaller Larger Smaller Larger Smaller

RV5-95 4.3 8.4 7.18 10.4 2.34 2.55

RV25-75 2.1 2.4 1.8 2.2 1.5 1.6

CSV 0.40 0.54 0.95 0.97 0.11 0.11

EB 0.29 0.31 0.43 0.31 0.08 0.08