The Incorporation of Meta-Analysis Results into Evidence-Based Decision Modelling
description
Transcript of The Incorporation of Meta-Analysis Results into Evidence-Based Decision Modelling
The Incorporation of Meta-Analysis Results into Evidence-Based
Decision Modelling
Nicola Cooper, Alex Sutton,
Keith Abrams, Paul Lambert, David JonesDepartment of Epidemiology & Public Health,
University of Leicester.
CHEBS, Multi-Parameter Evidence Synthesis Workshop, Sheffield, March 2002
Where we fit in with Tony’s intro
• Process Model relationship between evidence & parameters
– Consistency check
• Uncertainty Panacea Statistical error
– ½ Evidence relates to parameters indirectly
– Systematic errors
– Data quality, publication bias, etc
1) Pooled estimates
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
METHODOLOGIC PRINCIPLE
1) Pooled estimates
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
2) Distribution
METHODOLOGIC PRINCIPLE
1) Pooled estimates
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
3) Transformation of distribution to transition probability (if required)
2) Distribution
(i) time variables:
(ii) prob. variables:
j
ttP j)],(1ln[exp1
0
jjo ttP /1)],(1[1
METHODOLOGIC PRINCIPLE
1) Pooled estimates
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
3) Transformation of distribution to transition probability (if required)
2) Distribution
4) Application to model
(i) time variables:
(ii) prob. variables:
j
ttP j)],(1ln[exp1
0
jjo ttP /1)],(1[1
Respond
Stable
Progressive
Death
METHODOLOGIC PRINCIPLE
1) Net Clinical Benefit Approach
• Warfarin use for atrial fibrillation
2) Simple Economic Decision Model
• Prophylactic antibiotic use in caesarean section
3) Markov Economic Decision Model
• Taxane use in advanced breast cancer
1) Net Clinical Benefit Approach
• Warfarin use for atrial fibrillation
2) Simple Economic Decision Model
• Prophylactic antibiotic use in caesarean section
3) Markov Economic Decision Model
• Taxane use in advanced breast cancer
EXAMPLES
• Bayesian methods implemented using Markov Chain Monte Carlo simulation within WinBUGS software
• Random effect meta-analysis models used throughout• All prior distributions intended to be ‘vague’ unless
otherwise indicated• Where uncertainty exists in the value of parameters
(i.e. most of them!) they are treated as random variables
• All analyses (decision model and subsidiary analyses) implemented in one cohesive program
MODELLING ISSUES COMMON TO
ALL EXAMPLES
EXAMPLE 1: NET (CLINICAL) BENEFIT
Net Benefit = (Risk level x Risk reduction) – Harm
• Glasziou, P. P. and Irwig, L. M. An evidence based approach to individualizing treatment. Br.Med.J. 1995; 311:1356-1359.
Risk
Red
uctio
n in
abs
olut
e ris
k (b
enef
it)
Exc
ess
abso
lute
ris
k (h
arm
)
Harm
Benefit
Threshold
• Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin
• However, there is a risk of a fatal hemorrhage as a result of taking warfarin
• For whom do the benefits outweigh the risks?
RE-ANALYSIS OF WARFARIN FOR NON-
RHEUMATIC ATRIAL FIBRILLATION
1) Perform a meta-analysis of the RCTs to estimate the relative risk for benefit of the intervention
2) Use this to check the assumption that RR does not vary with patient risk
3) Check harm (adverse events) is constant across levels of risk (use RCTs and/or data from other sources) & estimate this risk
4) Place benefit & harm on same scale (assessment of QoL following different events)
5) Apply model - need to predict patients risk (identify risk factors and construct multivariate risk prediction equations)
METHOD OUTLINE
SOURCES OF EVIDENCE
Net Benefit
= (risk of stroke x relative reduction in risk of stroke)
- (risk of fatal bleed x outcome ratio)
Multivariate riskequations M-A of RCTs
M-A of RCTs/obsstudies
QoL study
Risk of embolic stroke in control arm (%/year)
Re
du
ctio
n in
ab
solu
te r
isk
of
em
bo
lic s
tro
ke (
%/y
ea
r)
4 6 8 10 12
02
46
81
0
Embolic stroke
Intracranial haemorrhage
02
46
81
0E
xce
ss a
bso
lute
ris
k o
f in
tra
cra
nia
l ha
em
orr
ha
ge
(%
/ye
ar)
Singer,D.E. Overview of the randomized trials to prevent stroke in atrial fibrillation. Ann Epidemiol 1993;3:567-7.
EVALUATING THE TRADE-OFF BETWEEN STROKE
AND HEMORRHAGE EVENTS IN TERMS OF QOL
• QoL following a fatal bleed = 0• Data available on QoL of patients following
stroke
– Glasziou, P. P., Bromwich, S., and Simes, R. J. Quality of life six months after myocardial infarction treated with thrombolytic therapy. The Medical Journal of Australia. 1994; 161532-536
Proportion with index greater than horizontal axis value
Time trade-off index
EVALUATION OF NET BENEFIT
(risk of stroke relative reduction in risk of stroke)
-
(risk of fatal bleed outcome ratio)
=
Net Benefit
Multivariate riskequations
Meta-analysesof RCTs
Meta-Analysis of RCTs /obs studies QoL study
0.002 0.004 0.006 0.008 0.010 0.012 0.014
050
100
150
200
250
300
risk of bleed per year
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
810
-1.5 -1.0 -0.5 0.0 0.5 1.0
02
46
reduction in relative risk
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.4
Outcome ratio
Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents) No.
Clinical risk
factors
No. of
patients
% of
cohort
Thrombo -embolism rate (% per year
(95% CI))
Mean (s.e.)
Median (95% CrI)
Probability of Benefit > 0
Simulated PDF
2 or 3
68
12
17.6 (10.5 to 29.9)
- 0.0004 (0.15)
0.06 (- 0.29 to 0.20)
54.2 %
-0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4
01
23
45
6
2 or 3 Clinical factors
Relative risk reduction for strokes takingwarfarin (1-RR): 0.23 (0.13 to 0.41)
Outcome ratio (1/QoLreduction) Median 3.75 (1.07 to 50), Mean 26.14,indicating the number of strokes that are equivalent to one death
Risk of stroke per year e.g. for 1 or 2 clinical risk factors: 6.0% (4.1 to 8.8)
Risk of fatal bleed per year takingwarfarin : 0.52% (0.27 to 0.84)
Risk of Stroke (Rate % per year)
Net
Ben
efit
(Str
oke
Equ
ival
ents
)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
Probability of benefit > 0.95
0 1 or 2 >=3
0 1 2 or 3 No. of clinical risk factors
No. of combined risk factors
MeanMedian
“TAKE-HOME” POINTS 1
Net-benefit provides a transparent quantitative framework to weigh up benefits and harms of an intervention
Utilises results from two meta-analyses and allows for correlation induced where studies included in both benefit and harm meta-analyses
Credible interval for net benefit can be constructed allowing for uncertainty in all model parameters
Use of prophylactic antibiotics to prevent wound infection following caesarean section
EXAMPLE 2: SIMPLE DECISION TREE
No infection (1-p2) Cost with antibiotics
Yes
Infection (p2) Cost with antibiotics + Cost of treatment
Prophylactic antibiotics?
No infection (1-p1) Cost with no antibiotics
No
Infection (p1) Cost of treatment
1) Cochrane review of 61 RCTs evaluating prophylactic antibiotics use for caesarean section
2) Event data rare: use “Exact” model for RR 3) Meta-regression: Does treatment effect vary with patients’
underlying risk (pc)?
ln(RRadjusted ) = ln(RRaverage)+ [ln(pc) - mean(ln(pc))]4) Risk of infection without treatment from ‘local’ hospital
data (p1)5) Derive relative risk of treatment effect for ‘local’ hospital
(using regression equation with pc=p1)6) Derive risk of infection if antibiotics introduced to ‘local’
hospital (p2)
p2 = p1 * RRadjusted
METHOD OUTLINE
UNDERLYING BASELINE RISK
ln(R
ela
tive
Ris
k)
ln(control group risk) centred on mean)
ln(relative risk) fit
-2.5 -2 -1.5 -1 -.5 0 .5 1 1.5
-3
-2.5
-2
-1.5
-1
-.5
0
.5
1
1.5
2 =0.24 (-0.28 to 0.81)
Local hospital event rate
No treatment effect
rr[1] sample: 20000
0.1 0.2 0.3 0.4 0.5
0.0 2.5 5.0 7.5 10.0
p1 sample: 20000
0.025 0.075 0.1 0.125
0.0
20.0
40.0
60.0
p2[1] sample: 20000
0.0 0.02 0.04
0.0 25.0 50.0 75.0 100.0
Mean (95% Credible Interval)
Posterior distribution
Relative Risk, RRadjusted 0.30 (0.21 to 0.40)
Prob(wound infection/placebo), p1
0.08 (0.06 to 0.10)
Prob(wound infection/antibiotics), p2
0.02 (0.015 to 0.034)
RESULTS
nwdreduct[1] sample: 20000
20.0 40.0 60.0 80.0
0.0
0.02
0.04
0.06
RESULTS
diff[1] sample: 20000
-150.0 -100.0 -50.0
0.0 0.01 0.02 0.03 0.04
tau.squared[1] sample: 20000
0.0 0.5 1.0 1.5
0.0
1.0
2.0
3.0
Mean (95% Credible Interval)
Posterior distribution
Reduction in cost using antibiotics
-£49.53 (-£77.09 to
-£26.79)
Number of wound infections avoided using antibiotics per 1,000
53.09(42.12 to
73.37)
Between study variance (random effect in M-A), 2
0.30 (0.05 to 0.74)
RESULTS (cont.)
COST-EFFECTIVENESS PLANE
Control dominates
Treatment less effective & less costly
Treatmentdominates
Treatment more effective & more costly
-140
-120
-100
-80
-60
-40
-20
0
20
-20 0 20 40 60 80 100
Number of wound infections avoided per 1,000 caesarean sectionsC
ost
diff
ere
nce
SENSITIVTY OF PRIORS
[1] Gamma(0.001,0.001) on 2
[2] Normal(0,1.0-6) truncated at zero on
[3] Uniform(0,20) on
[1]
[2]
[3]
caterpillar plot
Cost difference -80.0 -60.0 -40.0 -20.0
“TAKE-HOME” POINTS 2
Incorporates M-A into a decision model adjusting for a differential treatment effect with changes in baseline risk
Meta-regression model takes into account the fact that covariate is part of the definition of outcome
Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propogated
Sensitivity of overall results to prior distribution placed on the random effect term in a M-A
EXAMPLE 3: USE OF TAXANES FOR 2ND LINE
TREATMENT OF BREAST CANCER
Stages 1 & 2(cycles 1 to 3)
Stage 3(cycles 4 to 7)
Stage 4(cycles 8 to 35)
In 2nd line treatment
Respond Stable Progressive Dead
Respond Stable Progressive Dead
Respond Stable Progressive Dead
Treatment cycles
Post -Treatment
cycles
1) Define structure of Markov model2) Identify evidence used to inform each model
parameter using meta-analysis where multiple sources available
3) Transform meta-analysis results, where necessary, into format required for model (e.g. rates into transition probabilities)
4) Informative prior distributions derived from elicited prior beliefs from clinicians
5) Evaluate Markov model
METHOD OUTLINE
META-ANALYSES No. of
studies Time in weeks
(95% Credible Interval) Progression-free time 3 25 (15 to 24)
Time to response from stable 1 12 (6 to 18) Time to progressive from response 1 35 (29 to 41)
Overall survival time 3 53 (35 to 74) Probabilities
Response rate 4 0.43 (0.29 to 0.58) % moving directly to progressive at stage 2. 1 0.13 (0.08 to 0.18)
% with infections / febrile neutropenia 3 0.18 (0.04 to 0.56) % hospitalised with infection / febrile neutropenia 1 0.08 (0.05 to 0.11)
% dying from infections / febrile neutropenia 1 0.01 (0.00 to 0.02) % discontinue treatment due to adverse event 3 0.16 (0.03 to 0.49)
% with Neutropenia grades 3 & 4 2 0.94 (0.82 to 0.98) % with Anaemia grades 3 & 4 2 0.03 (0.00 to 0.28) % with Diarrhoea grades 3 & 4 3 0.09 (0.06 to 0.14) % with Stomatis grades 3 & 4 3 0.08 (0.04 to 0.14) % with vomiting grades 3 & 4 2 0.03 (0.00 to 0.12)
% with fluid retention grades 3 & 4 3 0.05 (0.02 to 0.12) % with cardiac toxicity grades 3 & 4 1 0.00 (0.00 to 0.02)
TRANSITION PROBABILITIES
Transition Probabilities (95% Credible Interval)
Infection/FN 0.09 (0.02 to 0.32)
Hospitalised due to infection/FN 0.04 (0.03 to 0.05)
Dying from infection/FN after hospitalisation 0.00 (0.00 to 0.01)
Discontinuation due to major adverse events 0.04 (0.04 to 0.16)
Adverse events – Neutropenia 0.50 (0.34 to 0.63)
Adverse events – Anaemia 0.01 (0.00 to 0.07)
Adverse events – Diarrhoea 0.02 (0.01 to 0.37)
Adverse events – Stomatis 0.02 (0.01 to 0.04)
Adverse events – Vomiting 0.01 (0.00 to 0.03)
Adverse events – Fluid retention 0.01 (0.00 to 0.03)
Adverse events – Cardiac toxicity 0.00 (0.00 to 0.01)
Transition directly to ‘progressive’ state 0.12 (0.08 to 0.18)
Transition ‘stable’ to ‘stable’ 0.65 (0.44 to 0.75)
Transition ‘stable’ to ‘response’ 0.16 (0.11 to 0.28)
Transition ‘stable’ to ‘progressive’ 0.18 (0.11 to 0.37)
Transition ‘response’ to ‘response’ 0.94 (0.93 to 0.95)
Transition ‘response’ to ‘progressive’ 0.06 (0.05 to 0.07)
Transition ‘progressive’ to ‘progressive’ 0.93 (0.79 to 0.96)
Transition ‘progressive’ to ‘death’ 0.07 (0.04 to 0.21)
1) Pooled estimates
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
3) Transformation of distribution to transition probability (if required)
2) Distribution
4) Application to model
(i) time variables:
(ii) prob. variables:
j
ttP j)],(1ln[exp1
0
jjo ttP /1)],(1[1
Respond
Stable
Progressive
Death
METHODOLOGIC PRINCIPLE
ELICITATION OF PRIORS
e.g. Response RateTaxane
x
x
x x x
x x x x x
x x x x x
x x
x x
x
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Standard
x
x
x x x x
x x x x x
x x x x x
x x
x x
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
RESPONSE RATE
Response rate (docetaxel)
9585756555453525155
50
40
30
20
10
0
Std. Dev = 17.28
Mean = 38
N = 280.00
logit (Response rate for docetaxel)
3.5
3.0
2.5
2.0
1.5
1.0.5-.0
-.5
-1.0
-1.5
-2.0
-2.5
-3.0
-3.5
80
60
40
20
0
Std. Dev = .87
Mean = -.6
N = 280.00
Response rate for doxorubicin
85756555453525155
50
40
30
20
10
0
Std. Dev = 14.75
Mean = 31
N = 300.00
logit (Response rate for doxorubicin)
2.5
1.5.5-.5
-1.5
-2.5
-3.5
-4.5
-5.5
-6.5
-7.5
100
80
60
40
20
0
Std. Dev = .92
Mean = -1.0
N = 300.00
Bayesian (MCMC) Simulations
-£4,000
-£2,000
£0
£2,000
£4,000
£6,000
£8,000
£10,000
-0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50
Incremental utility
Incr
eme
nta
l co
st
Doxorubicin dominates
Docetaxel more effective but more costly
Docetaxel less costly but less
effective
Docetaxel dominates
COST-EFFECTIVENESS PLANE
“TAKE-HOME” POINTS 3
Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in a unified framework (facilitating sensitivity analysis)
Provides a framework to incorporate prior beliefs of experts
FURTHER ISSUES
• Handling indirect comparisons correctly•E.g. Want to compare A v C but evidence only available on A v B & B v C etc.•Avoid breaking randomisation
• Necessary complexity of model?•When to use approaches 1,2,3 above?
• Use of predictive distributions•Necessary when inferences made at ‘unit’ level (e.g. hospital in 2nd example) rather than ‘population’ level?
• Incorporation of EVI
• Handling indirect comparisons correctly•E.g. Want to compare A v C but evidence only available on A v B & B v C etc.•Avoid breaking randomisation
• Necessary complexity of model?•When to use approaches 1,2,3 above?
• Use of predictive distributions•Necessary when inferences made at ‘unit’ level (e.g. hospital in 2nd example) rather than ‘population’ level?
• Incorporation of EVI
MODEL SPECIFICATION
)(exp ),(~
))log(,min()log( )log(
1,.....,61 ),(~ ),(~
2 ΔRRΔNormal
ppp
ipnBinomialrpnBinomialr
santibiotici
ciii
ti
cii
ti
ti
ti
ci
ci
ci
Warn et al 2002 Stats in Med (in press)
Bayesian random effects M-A model specification: ln(RR)
)001.0,001.0(~ (0,0.1)~
)100,1(~ )100,1(~ ),(~
2 maInverseGamτNormalΔ
UniformUniformBetapci
Prior distributions: