© Nuffield Trust

LA FRAGILITA : DAI MODELLI TEORICIALLA VALUTAZIONE DELLEESPERIENZE

IBKOST: OD TEORETICNIHMODELOV DOOCENJEVANJA IZKU ENJ

Dr Martin Bardsley Ravenna, 12 ot tobre 2012

Predictive Models for Health and Social CareProgetto strategico "E-health /Strate kega projekta "E-heath"

© Nuffield Trust

Promote independent analysis and informed debate on healthcare policy across the UKCharitable organization founded in 1940Formerly a grant-giving organizationSince 2008 we have been conducting in-house research and policy analysisSignificant interest in uses of predictive risk techniques

The Nuffield Trust

William Morris1st Viscount Nuffield

(1877 -1963)

Progetto e-health

© Nuffield Trust© Nuffield Trust

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

NHS (UK)

England: 10 strategic health authorities

Populations: NHS England: 51.2m; NHS Scotland: 5m;NHS Wales: 2.9m;NHS N Ireland: 1.7m

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Background

Comprehensive benefits

Free at the point of use (copayments minimal)

UK health care spending (2007-08) 9.4 % GDP (1% private, 8.4% public)

UK NHS (2007-08)£105.6 billion (approx £1730 per capita )

Progetto e-health

New funding arrangements

Progetto e-health

UK health spending as share of GDP

Progetto e-health

Are emergency admissions rising?

Reproduced from Trends in emergency admissions in England 2004-2009: is greater efficiency breeding inefficiency

Possible reasons

Ageing population

Public expectations

Care of frail older people

Defensive medicine

Central targets / payment by results

Changes in other linked services

Over reliance on A&E for urgent careNumber of emergency admissions in England 1996-2009,

with period investigated marked in red

Progetto e-health

© Nuffield Trust16 October 2012 © Nuffield Trust

Progetto e-health

© Nuffield Trust

Uses of predictive risk techniques

Predictive modelling aims to identify people at risk of future event

Progetto e-health

© Nuffield Trust

Introduction of predictive modelling to UK

BMJ in paper in 2002 showed KaiserPermanente in California seemed to provide higher quality healthcare than the NHS at lower cost.

Kaiser identify high risk people in their population and manage them intensively to avoid admissions

Follow up paper noted chief executive of a managed care organisation commented: Without case management, we are sunk in

the marketplace.

Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143

Can the NHS learn from US managed care organisations? BMJ 2004;328:223-225

Progetto e-health

© Nuffield Trust

Predictive modelling is only as effective as the intervention itis used to trigger

Case Management

Intensive Disease Management

Less Intensive Disease Management

Wellness Programmes

Top 0.5%

0.5 5.0%

6 - 20%

21 100%

Providers need to know potential costs of the outcome to build business case for intervention

Progetto e-health

© Nuffield Trust

To prevent, we need to know who will be an intensive user in the future

Predictive models try to identify people here

It s not the people who are current intensive users

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Progetto e-health

© Nuffield Trust

New research last year showed that neither doctors, nurses nor case managers were able to predict which patients were at highest risk of readmission to hospital.

An alternative approach to clinicians predicting hospital admissions is to use statistical models instead.

Why use predictive risk to find cases?

(Allaudeen et al, Journal of General Internal Medicine, 2011)

Progetto e-health

© Nuffield Trust

Population level screening for high risk individivuals

A predictive risk model tries to

sort it out

At the start of the year, no one knows who s who

Progetto e-health

© Nuffield Trust

Introduction of predictive modelling to UK

Patterns in routine data identify high-risk people next year

Relies on exploiting existing information:+ve: systematic; not costly data collections; fit into existing systems-ve: information collected may not be predictive

Use pseudonymous, person-level data

In health sector a number of predictive models are available e.g. PARR++ and the combined model.

Progetto e-health

© Nuffield Trust

Developing a predictive risk model

Progetto e-health

© Nuffield Trust

Protecting individuals identities

Progetto e-health

© Nuffield Trust

Developing a predictive risk model

Progetto e-health

© Nuffield Trust

In 2006, the Department of Health (DH) invested in two predictive models (or risk stratification tools ) for the NHS in England.

PARR widely used by NHS (because software was free and SUS data only)

Predicts readmission in next year PPV 65%

Designed to be run by PCTs periodically, requires up-to-date diagnostic codes

A predictive risk tool: PARR

Hospital provides SUS

PCT runs PARR++

Patients selected for intervention (via GP)

Progetto e-health

© Nuffield Trust

Key metrics for performance of a model (PPV and sensitivity)

Sensitivity

Positive predictive value

Trade-off between PPV and sensitivity sometimes summarised as the c-statistic

Not very intuitive, and reflects average performance across all risk levels

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Model Risk threshold PPV (%) Sensitivity (%)PARR (England) 50 65.3 54.3

70 77.4 17.880 84.3 8.1

SPARRA (Scotland) 50 59.4 18.070 76.1 3.3

S Care model (Pooled £1K)

50

70

55

60

19

10

Typical accuracy models currently used to predict hospital admission

Progetto e-health

© Nuffield Trust

Emerging market in England

August 2011, the Department of Health

announced that it had no plans to

commission national updates of the

latest Patients at Risk of Re-

hospitalisation tool (PARR++) or the

Combined Predictive Model

Range of new/established commercial

organisation developing risk tools

Creation of new commissioning groups

and new markets

Increasing ease of accessing GP data

Continuing financial pressures and the

search for ways to reduce emergency

hospital care.

SPARRA PARR (++)

SPARRA MD Combined Predictive Model

PRISM PEONY

AHI Risk adjuster LACE

ACGs (Johns Hopkins) MARA (Milliman Advanced Risk Adjuster)

DxCGs (Verisk) Dr Foster Intelligence

SCOPELACE

QResearch models eg QD score

RISC

Variants on basic admission/readmission predictions:Short term readmissions Social careCondition specific tools costs

Progetto e-health

© Nuffield Trust

Examples of some models used in UK

Readmission to hospital within 1 year eg PARR, SPARRA

Readmission within 30 days eg LACE, PARR30

Admission to hospital eg Combined Predictive Model, ACGs,PRISM

Move into intensive social care eg Nuffield Model

Likelihood of chronic disease eg QRISK models diabetes

Hospital costs in the coming year eg PBRA

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

16 October 2012 © Nuffield Trust

Authors: Bardsley M, Billings J, Dixon J, Georghiou T, Lewis GH, Steventon A (2011) Predicting who will use intensive social care: case finding tools based on linked health and social care data , Age and Ageing 40(2): 265-270

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Information flows

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Data linkage Social & secondary care interface

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Which variables are important in pooled £1k model?

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Models using lower £1k thresholds

Progetto e-health

© Nuffield Trust16 October 2012 © Nuffield Trust

Progetto e-health

© Nuffield Trust

From models to tools

A predictive risk tool has three parts:

The model

The software

The data

Progetto e-health

© Nuffield Trust

A predictive risk tool has three parts:

The model

The software

The data

From models to tools

Range of models is growing all the time from academic groups and from proprietary information tools

Many models available for free

Some PCTs developed their own

Range of commercial companies offering complete tools + technical support in marshalling data.

Models can be applied using a standard database package (Business Objects, SQL Server even MS Access)

Some business intelligence packages now come with predictive risk modelling built-in

Many PCTs have already created data-warehouses

Progetto e-health

© Nuffield Trust

Accident and emergency 350,000 records

Outpatients1,680,000 records

Inpatients360,000 records

Social care240,000 records

Community matrons20,000 records

GPs60 practices48.5 million records

Relative size of data sets collectedFor one WSD area

March 2011

Progetto e-health

© Nuffield Trust

Using the data available

Progetto e-health

© Nuffield Trust

Testing for gaps in care

Progetto e-health

© Nuffield Trust16 October 2012 © Nuffield Trust

Progetto e-health

© Nuffield Trust

Design and implementation

Model development limitationsPredict risks of expensive things you think you can do something about

Make sure your data base has most of the key risk factors

Recognize the trade-offs between model accuracy and sensitivity

Intervention design flawsDesign the intervention after the risk model has been developed

Use data from model development to help design the intervention

Recognize you are probably going to need more information

Intervention implementation flawsRoll it out in at least quasi-experimental mode

Track dosage levels (who does what to whom and how)

Avoid enrollment criteria leakage

Evaluate impact of the intervention as rigorously as possible

Ref. Prof John Billings

Progetto e-health

© Nuffield Trust

Community matrons

March 2011

General population

Top 0.5%

0.5% - 5%

5% - 20%

20% - 100%

Very high risk

High risk

Moderate risk

Low risk

Progetto e-health

Virtual Ward BCommunity Matron

Nursing complementHealth VisitorWard ClerkPharmacist

Social WorkerPhysiotherapist

Occupational Therapist Mental Health Link

Voluntary Sector Link

Virtual Ward ACommunity Matron

Nursing complementHealth VisitorWard ClerkPharmacist

Social WorkerPhysiotherapist

Occupational TherapistMental Health Link

Voluntary Sector Link

Specialist Staff

Specialist nursesAsthmaContinenceHeart Failure

Palliative care team

Alcohol service

Dietician

GP Practice 1

GP Practice 2

GP Practice 3

GP Practice 5

GP Practice 4

GP Practice 6

GP Practice 7

GP Practice 8

Progetto e-health

© Nuffield Trust

Telehealth and telecare

Images are the copyright of Tunstall Group Ltd

Progetto e-health

© Nuffield Trust

Distribution of Combined Model risk scoresImportance of risk adjustment

March 2011

General population

Top 0.5%

0.5% - 5%

5% - 20%

20% - 100%

Telehealth trial participants

Top 10%

10% - 45%

45% - 85%

85% - 100%

Very high risk

High risk

Moderate risk

Low risk

Progetto e-health

© Nuffield Trust

When can I expect to see a return on investment?

(How accurate is the model? How effective is the intervention? How much does it cost?)

Savings are linked to cost of intervention and its effectiveness

Example: Average costs of readmission for high

risk patients are ~£1000Intervention reduces readmission by

10%Then intervention has to cost less

than £100 to save money

Progetto e-health

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Summary

Range of predictive modelling tools have been developed

Focus is on prioritising cases for preventive care with the expected benefits of reducing the increasing demand for emergency hospital care

Models can span both health and social care

Technical details of model performance is important but so how is the way the model is implemented

Significant challenges in organising data but linked data sets can offer additional benefits

Range of applications not fully tested

Progetto e-health

© Nuffield Trust16 October 2012

www.nuffieldtrust.org.uk

Sign-up for our newsletterwww.nuffieldtrust.org.uk/newsletter

Follow us on Twitter:Twitter.com/NuffieldTrust

© Nuffield Trust

Insert presenter s email address here

Progetto e-health

http://www.nuffieldtrust.org.ukhttp://www.nuffieldtrust.org.uk/newsletter

© Nuffield Trust

Grazie per lGrazie per l attenzione!attenzione!

Hvala za vaHvala za va o pozornost!o pozornost!

Dr Martin Bardsley

Head of Research

Nuffield Trust ,London

Progetto finanziato nell'ambito del Programma per la Cooperazione TransfrontalieraItalia-Slovenia 2007-2013, dal Fondo europeo di sviluppo regionale e dai fondi nazionaliProjekt sofinanciran v okviru Programa ezmejnega sodelovanja Slovenija-Italija 2007-2013 iz sredstev Evropskega sklada za regionalni razvoj in nacionalnih sredstev

Ministero dell'Economia e delle Finanze

LA FRAGILITA : DAI MODELLI TEORICIALLA VALUTAZIONE DELLEESPERIENZE Ravenna, 12 ot tobre 2012

in collaborazione con /v sodelovanju z