Methods for the Estimation of the NICE Cost Effectiveness ...

Methods for the Estimation of the

NICE Cost Effectiveness

Threshold Karl Claxton,1,2 Steve Martin,2 Marta Soares,1 Nigel Rice,1,2 Eldon Spackman,1 Sebastian Hinde,1 Nancy

Devlin,3 Peter C Smith,4 Mark Sculpher1

1. Centre for Health Economics, University of York, UK

2. Department of Economics and Related Studies, University of York, UK

3. Office of Health Economics, London, UK

4. Imperial College, London, UK

What do we need?

• Compare

– Health expected to be gained

– Health expected to be lost due to additional NHS costs

• Expected health effects of changes in NHS expenditure

• What its not

– Consumption value of health (willingness to pay)

– Marginal productivity of ‘ideal’ NHS

• No simple relationship with changes in

– Budget, prices and productivity

– Health production outside NHS

How can we estimate it?

• Implied value from past decisions based on informal judgements

NICE threshold

Range 2004 (2001)

Appleby et al 2007

Martin et al 2008, 2009

• Find out what decisions are made and estimate impact on cost and health – Which/ whose decisions?

– Caused by NICE guidance?

– Effect of decisions on health and costs?

– But we don't need to know which decisions just the health effects

• Estimate the relationship between changes in expenditure and outcomes

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in

expenditure allocated to

PBCs?

11 PBCs

Cost per death averted

PBC without mortality

signal

11 PBCs

How does a change in PBC expenditure effect PBC mortality?

11 PBCs

How can we estimate effects of expenditure on

mortality ?


mortality (deaths)?

• Change in PBC expenditure due to change in overall expenditure

(all 23 PBCs)

– Differences in spending on a particular PBC and total spend across PCTs

– Account for other reasons why PBC spend might differ between PCTs

– Isolate the effects on PBC spend of changes in overall expenditure

• Change in PBC mortality (deaths) due to change in PBC expenditure

(11 PBCs)

– Differences in PBC mortality and PBC expenditure across PCTs

– Account for other reasons why PBC mortality might differ between PCTs

– Isolate the effects on PBC mortality of changes in PBC expenditure

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs


mortality?

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs


Health effects of changes in

expenditure at same rate as other 11PBCs


signal

11 PBCs


11 PBCs

PBC 23 GMS


mortality?

Estimates of the threshold (2008-09)

Cost per death

averted

Cost per life year

Cost per QALY (mortality effects)

Cost per QALY

Qol associated with LYs - 1 Norms Based on burden

Qol during disease - 0 0 Based on burden

YLL per death averted - 4.5 YLL 4.5 YLL 4.6 YLL

QALYs per death averted - 4.5 YLL 3.8 QALY 12.7 QALY

11 PBCs (with mortality)

£105,872

£23,360

£28,045 £8,308

All 23 PBCs

£114,272

£25,214

£30,270 £18,317

PBC 23 GMS

How can we estimate effects on life years C

han

ge in

ove

rall

exp

end

itu

re

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs

PBC 23 GMS


han

ge in

ove

rall

exp

end

itu

re

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs

What are the LY effects of

changes in PBC mortality?

11 PBCs

Cost per life year

What are the life year effects of changes in PBC

mortality?

• Effects on all deaths within a PBC (group of ICD codes)

– Not all deaths are reported by PCT (all ICD codes)

– Apply % effects (observed) to deaths in all ICD codes in PBC (ONS)

• What years of life are lost due to mortality?

– LE of the age and gender distribution in each ICD within the PBC

– Age of death compared to LE

– Account for all deaths below LE and above LE

• Accounts for deaths from other causes

– Death averted faces the mortality risk of a matched population

PBC 23 GMS


han

ge in

ove

rall

exp

end

itu

re

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year

PBC 23 GMS


han

ge in

ove

rall

exp

end

itu

re

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year

PBC 23 GMS

11 PBCs




Cost per death

averted

Cost per life year


Cost per QALY






£105,872

£23,360

£28,045 £8,308

All 23 PBCs

£114,272

£25,214

£30,270 £18,317

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year

How can we adjust life years for quality?

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year

Adjust life year effects for

quality

11 PBCs

Cost per QALY (mortality)


0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 20 40 60 80 100

Qo

L sc

ore

Age

norm males

diseased males

Adjusting life year effects for quality

• Assumptions are relaxed using measures of burden Quality of life for the general population by

age and gender

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 20 40 60 80 100

Qo

L s

core

Age

males

females

• Life years lived at Qol norms by age and gender – All disease is acute or symptoms are ‘curable’

• Life years lived with Qol of disease (decrement to norms)

– All disease is chronic (life long) and ‘incurable’

Quality of life for males in PBC1

(infectious disease

PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year


quality

11 PBCs



PBC 23 GMS

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs



signal

11 PBCs


11 PBCs



11 PBCs

Cost per life year


quality

11 PBCs


PBC 23 GMS

11 PBCs





Cost per death

averted

Cost per life year

Cost per QALY (mortality effects only)

Qol associated with LYs - 1 Norms Disease

Qol during disease - 0 0 0



11PBCs (with mortality)

£105,872

£23,360

£28,045 £35,397

All 23 PBCs

£114,272

£25,214

£30,270 £38,206

How can we account for possible effects on

quality of life?

• No observations of quality life by PBC at PCT level

– Quality of life is important in 11 PBCs with mortality

– Mortality is (almost) irrelevant in the other 11 PBCs

– Much NHS activity is primarily to improve quality of life

• Possible responses

– Assume that NHS expenditure has no effects on quality of life

– Use what can be observed to impute what cannot

• Effects on quality of life in 11 PBCs with mortality

• QALY effects in the other 11 PBCs

– Use all the information we have about the other 11 PBCs

PBC 23 GMS

How can we account for possible effects

on quality of life?

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs


signal

11 PBCs


11 PBCs 11 PBCs

% effect of a change in PBC expenditure

on burden of disease (LY)

PBC 23 GMS


on quality of life?

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs


signal

11 PBCs


11 PBCs 11 PBCs



PBC 23 GMS

11 PBCs

Same % effect on burden of

disease (totals from

the other 11PBCs)

PBC 23 GMS


on quality of life?

Ch

ange

in o

vera

ll ex

pen

dit

ure

How are changes in


PBCs?

11 PBCs


signal

11 PBCs


11 PBCs 11 PBCs



PBC 23 GMS

11 PBCs

Same % effect on burden of

disease (totals from

the other 11PBCs)

PBC 23 GMS

Measures of QALY burden

of disease

Life years (ONS)

Quality of life

(HoDAR MEPS)

Age, gender and duration

of disease (GBD)

Cost per QALY (life year and

quality effects)


Cost per death

averted

Cost per life year


Cost per QALY






£105,872

£23,360

£28,045

£8,308

All 23 PBCs

£114,272

£25,214

£30,270

£18,317

What are the expected health consequences of £10m?

Change in spend (£000) Additional deaths LY lost Total QALY lost Due to premature death Quality of life effects

Totals £10,000 37 167 546 107 439

Cancer £324.000 3 27 19 18 1

Circulatory £550.000 17 84 78 53 25

Respiratory £332.000 10 12 166 7 159

Gastro-intestinal £232.000 2 18 32 12 20

Infectious diseases £237.000 1 4 11 3 9

Endocrine £137.000 < 0.5 4 44 2 42

Neurological £433.000 1 5 79 3 76

Genito-urinary £336.000 2 2 8 1 6

Trauma & injuries* £558.000 0 0 0 0 0

Maternity & neonates* £495.000 < 0.05 < 0.5 0 < 0.5 < 0.5

Disorders of Blood £292.000 < 0.5 1 10 1 10

Mental Health £2,532.000 2 7 51 4 46

Learning Disability £147.000 < 0.5 1 2 < 0.5 1

Problems of Vision £275.000 < 0.05 < 0.5 4 < 0.5 3

Problems of Hearing £124.000 < 0.05 < 0.5 6 < 0.05 6

Dental problems £409.000 < 0.05 < 0.05 7 < 0.05 7

Skin £279.000 < 0.5 1 2 1 1

Musculo skeletal £514.000 < 0.5 2 25 1 24

Poisoning and AE £132.000 < 0.05 < 0.5 1 < 0.5 1

Healthy Individuals £501.000 < 0.05 < 0.5 0 < 0.05 < 0.5

Social Care Needs £426.000 0 0 0 0 0

Other (GMS) £735.000 0 0 0 0 0

Is it likely to be an under or over estimate?

• Health effects over estimated (threshold underestimated)? – Deaths averted returns the individuals to the mortality risk of the general

population (matched for age and gender)

– Small positive correlation between expenditure and outcome elasticities

– Apply estimates (data reported at PCT) to all PBC mortality

• Health effects under estimated (threshold overestimated)? – Mortality and quality of life effects restricted to one year

– No effects of prevention (reduce incidence into the at risk population)

– Effects of changes in GMS (and PBC22 & 16) expenditure not fully captured

– Share of changes in expenditure favours PBCs with high cost per QALY

• Other assumptions – Surrogacy

• Are % mortality effects a good surrogate for % Qol effects?

– Extrapolation

• Is the proportionate effect on QALY burden of changes in spend similar in the other PBCs?

Implied PBC cost per QALY

• Which PBCs matter most?

– Share of change in spend, share of health effects and how much

implied PBC cost per QALY differs from £18,317

– 11 PBCs where proportionate effects are imputed

• Mental health most important PBC (imputed cost per QALY £49,835)

• Evidence suggests cost per QALY of mental health interventions lower

• Differences in the implied PBC cost per QALY

– Misallocation of resources?

– Social value of health effects (maternity and neonates)

– Cannot observe quality of life effects at PCT level

• Quality of life effects not proportional to mortality effects

• Health effects more than proportional to QALY burden

– Effect on outcomes in other PBCs

k

k

k1

NB1

k2

NB2

k3

NB3

Implications of uncertainty in the estimate (Single threshold value that can be compared to an ICER)

Net

Ben

efit

0

Ek NB(k)

k*

Health

Impact of investment and disinvestment?

H1

-ΔE

B1

1/k1

1/k1-

+ΔE

k1

-ΔE 0 +ΔE

Threshold

k1+

k1-

1/k1+

Variation in expenditure

1/k1

£14,083 per LY

£8,441 per LY

£10,604 Per LY

Health

Budget

How does the threshold change with overall

expenditure?

H1

B1

1/k1

1/k1 Increase productivity

Current NHS 1/k1

B2 Waste

Only eliminate waste

1/k1

2007 2008

Nominal £18,624 £18,317

2007 NHS prices £18,624 £17,629

Summary of considerations

• On balance £18,317 is more likely to be an over than underestimate of the threshold – Upper bound of the NICE threshold is almost certainly too high

– Lower bound may also be too high

• Uncertainty in the estimate suggests a policy threshold set as less than the mean estimate

• Threshold less than the mean estimate when imposing costs on the NHS (reducing expenditure)

• No evidence of growth in threshold with increases in real budget and prices

• Some evidence that threshold more likely to fall rather than rise as NHS comes under more financial pressure

What type of data and research could improve

the estimate?

• Longer and more complex lag structure

– Duration of effect on mortality might be feasible (capture more health effects)

– Estimating life year effect of mortality more problematic

• Simultaneous estimation across PBCs

– Likely to capture more health effects

• Evolving PBC data (PCT and CCG boundaries)

• Extending measures of health outcome

– Analysis of PROMs data

– IAPT and mental health outcomes

• Incidence and duration of disease

– WHO GBD

– GPRD

Additional slides

• Reserve slides if needed during discussion

∆ LY

∆ Qol Surrogacy Quality of life effects

(each of 11PBCs where

can estimate LY effects)

% reduction in LY

burden

Qol norm

Qol dis

LY norm LY dis

Qol norm

Qol dis

LY norm LY dis

∆ LY

Extrapolation QALY effects

(other 11PBCs)

Same % effect on

burden

∆ Qol

Which PBCs matter most? PBC % spend % health Elasticity PBC cost per Qol

2 Cancer 3.24 3.50 0.35 £16,997

10 Circulatory 5.50 14.32 1.43 £7,038

11 Respiratory 3.32 30.45 3.05 £1,998

13 Gastro-intestinal 2.32 5.83 0.58 £7,293

1 Infectious diseases 2.37 2.08 0.21 £20,829

4 Endocrine 1.37 8.04 0.80 £3,124

7 Neurological 4.33 14.48 1.45 £5,480

17 Genito-urinary 3.36 1.40 0.14 £43,813

16 Trauma & injuries* 5.58 0 0 NA

18+19 Maternity & neonates* 4.95 0.03 0.00 £2,969,208

3 Disorders of Blood 2.92 1.89 0.19 £28,305

5 Mental Health 25.32 9.31 0.93 £49,835

6 Learning Disability 1.47 0.34 0.03 £78,854

8 Problems of Vision 2.75 0.66 0.07 £76,850

9 Problems of Hearing 1.24 1.19 0.12 £19,070

12 Dental problems 4.09 1.34 0.13 £55,916

14 Skin 2.79 0.29 0.03 £174,775

15 Musculo skeletal 5.14 4.65 0.47 £20,254

20 Poisoning and AE 1.32 0.15 0.01 £163,766

21 Healthy Individuals 5.01 0.06 0.01 £1,483,012

22 Social Care Needs 4.26 0 0 NA

23 Other 7.35 0 0 NA

How uncertain are the estimates?

An assessment of parameter uncertainty

Figure 5.1 Cumulative probability density function for the cost per QALY threshold

0.00

0.20

0.40

0.60

0.80

1.00

£0 £10,000 £20,000 £30,000 £40,000

Pro

bab

ilit

y

Cost per QALY threshold

11 PBCs

23 PBCs

Decomposing QALYs Table C.79: Decomposing estimated QALY effects by PBC (2008)

PBC

QALY change (total)

QALY change (death)

% QALY gained

for premature

death for disability while alive

[1] [2] [3] [4]

2 Cancer 1,453 1,393 96% 4% 10 Circulatory 5,125 4,054 79% 21% 11 Respiratory 10,947 758 7% 93% 13 Gastro-intestinal 2,087 1,024 49% 51%

1 Infectious diseases 14 9 67% 33% 4 Endocrine 2,921 269 9% 91% 7 Neurological 441 43 10% 90%

17 Genito-urinary 13 5 40% 60% 16 Trauma & injuries* 0 0 NA NA

18+19 Maternity & neonates* 22 7 30% 70%

3 Disorders of Blood 689 35 5% 95% 5 Mental Health 3,397 296 9% 91% 6 Learning Disability 125 25 20% 80% 8 Problems of Vision 240 9 4% 96% 9 Problems of Hearing 434 3 1% 99%

12 Dental problems 489 0 0% 100% 14 Skin 107 39 37% 63% 15 Musculo skeletal 1,697 84 5% 95% 20 Poisoning and AE 54 9 16% 84% 21 Healthy Individuals 23 4 16% 84% 22 Social Care Needs 0 0 NA NA 23 Other 0 0 NA NA

Methods for the Estimation of the NICE Cost Effectiveness ...

Documents

Transcript of Methods for the Estimation of the NICE Cost Effectiveness ...