Meta-analysis: summarising data for two arm trials and other simple outcome studies Steff Lewis...
-
Upload
clinton-blewitt -
Category
Documents
-
view
217 -
download
1
Transcript of Meta-analysis: summarising data for two arm trials and other simple outcome studies Steff Lewis...
Meta-analysis:summarising data for two
arm trials and other simple outcome studies
Steff Lewisstatistician
When can/should you do a meta-analysis?
• When more than one study has estimated an effect
• When there are no differences in the study characteristics that are likely to substantially affect outcome
• When the outcome has been measured in similar ways
• When the data are available (take care with interpretation when only some data are available)
Types of data• Dichotomous/ binary data
• Counts of infrequent events • Short ordinal scales
• Long ordinal scales
• Continuous data
• Censored data
What to collect
• Need the total number of patients in each treatment group
Plus:
• Binary data– The number of patients who had the
relevant outcome in each treatment group
• Continuous data– The mean and standard deviation of the
effect for each treatment group
• Then enter data into RevMan / MIX (easy to use and free)http://www.mix-for-meta-analysis.info/
http://www.cc-ims.net/RevMan/
Or R (harder to use and free)
Or Stata (harder to use and costs)
Etc etc....
Summary statistic for each study
• Calculate a single summary statistic to represent the effect found in each study
• For binary data– Risk ratio with rarer event as outcome
• For continuous data– Difference between means
Meta-analysis
Averaging studies
• Starting with the summary statistic for each study, how should we combine these?
• A simple average gives each study equal weight
• This seems intuitively wrong• Some studies are more likely to give an
answer closer to the ‘true’ effect than others
Weighting studies
• More weight to the studies which give us more information– More participants– More events– Lower variance
• Weight is closely related to the width of the study confidence interval: wider confidence interval = less weight
Displaying results graphically
• RevMan (the Cochrane Collaboration’s free meta-analysis software) and MIX produce forest plots (as do R and Stata and some other packages)
Review: Corticosteroids for acute traumatic brain injuryComparison: 01 Any steroid administered in any dose against no steroid Outcome: 01 Death at end of follow up period
Study Steroid Control RR (fixed) RR (fixed)or sub-category n/N n/N 95% CI 95% CI
Alexander 1972 16/55 22/55 0.73 [0.43, 1.23] Brackman 1983 44/81 47/80 0.92 [0.70, 1.21] CRASH 2004 1052/4985 893/4979 1.18 [1.09, 1.27] Chacon 1987 1/5 0/5 3.00 [0.15, 59.89] Cooper 1979 26/49 13/27 1.10 [0.69, 1.77] Dearden 1986 33/68 21/62 1.43 [0.94, 2.19] Faupel 1976 16/67 16/28 0.42 [0.24, 0.71] Gaab 1994 19/133 21/136 0.93 [0.52, 1.64] Giannotta 1984 34/72 7/16 1.08 [0.59, 1.98] Grumme 1995 38/175 49/195 0.86 [0.60, 1.25] Hemesniemi 1979 35/81 36/83 1.00 [0.70, 1.41] Pitts 1980 114/201 38/74 1.10 [0.86, 1.42] Ransohoff 1972 9/17 13/18 0.73 [0.43, 1.25] Saul 1981 8/50 9/50 0.89 [0.37, 2.12] Stubbs 1989 13/98 5/54 1.43 [0.54, 3.80] Zagara 1987 4/12 4/12 1.00 [0.32, 3.10] Zarete 1995 0/30 0/30 Not estimable
Total (95% CI) 6179 5904 1.12 [1.05, 1.20]Total events: 1462 (Steroid), 1194 (Control)Test for heterogeneity: Chi² = 26.46, df = 15 (P = 0.03), I² = 43.3%Test for overall effect: Z = 3.27 (P = 0.001)
0.1 0.2 0.5 1 2 5 10
Steroid better Steroid worse
Heterogeneity
What is heterogeneity?
• Heterogeneity is variation between the studies’ results
Causes of heterogeneityDifferences between studies with respect to:• Patients: diagnosis, in- and exclusion
criteria, etc.• Interventions: type, dose, duration, etc.
• Outcomes: type, scale, cut-off points, duration of follow-up, etc.
• Quality and methodology: randomised or not, allocation concealment, blinding, etc.
How to deal with heterogeneity
1. Do not pool at all
2. Ignore heterogeneity: use fixed effect model
3. Allow for heterogeneity: use random effects
model
4. Explore heterogeneity: meta-regression
(tricky)
How to assess heterogeneity from a forest plot
Statistical measures of heterogeneity
• The Chi2 test measures the amount of variation in a set of trials, and tells us if it is more than would be expected by chance
0.61 ( 0.46 , 0.81 )
Study
Liggins 1972
Pooled
Block 1977
Morrison 1978
Taeusch 1979
Papageorgiou 1979
Schutte 1979
Collaborative Group 1981
Estimates with 95% confidence intervals
0.05 0.25 4
Corticosteroids better Corticosteroids worse
Odds ratio
1
Heterogeneity test
Q = 11.2 (6 d.f.)
p = 0.08
Trials from Cochrane logo:
Corticosteroids for preterm birth
(neonatal death)
Heterogeneity test
Q = 44.7 (27 d.f.)
p = 0.02
Estimates with 95% confidence intervalsStudy
Liggins 1972
Block 1977
Morrison 1978
Taeusch 1979
Papageorgiou 1979
Schutte 1979
Collaborative Group 1981
0.05 0.25 4
Odds ratio
1 0.05 0.25 4
Odds ratio
1
Heterogeneity test
Q = 11.2 (6 d.f.)
p = 0.08
Corticosteroids for preterm birth
(neonatal death)
where Q = heterogeneity 2 statistic
I2 can be interpreted as the proportion of total variability explained by heterogeneity, rather than chance
I squared quantifies heterogeneity
Q
dfQI
1002
• Roughly, I2 values of 25%, 50%,
and 75% could be interpreted as indicating low, moderate, and high heterogeneity
• For more info see: Higgins JPT et al. Measuring inconsistency in meta-analyses. BMJ 2003;327:557-60.
Fixed and random effects
Fixed effect
Philosophy behind fixed effect model:• there is one real value for the treatment
effect• all trials estimate this one value
Problems with ignoring heterogeneity:• confidence intervals too narrow
Random effects
Philosophy behind random effects model:
• there are many possible real values for the treatment effect (depending on dose, duration, etc etc).
• each trial estimates its own real value
Example
Could we just add the data from all the trials together?
• One approach to combining trials would be to add all the treatment groups together, add all the control groups together, and compare the totals
• This is wrong for several reasons, and it can give the wrong answer
If we add up the columns we get 34.3% vs 32.5% , a RR of 1.06, a higher chance of death in the steroids group
From a meta-analysis, we getRR=0.96 , a lower chance of death in the steroids group
Problems with simple addition of studies
• breaks the power of randomisation• imbalances within trials introduce bias
*
The Pitts trial contributes 17% (201/1194) of all the data to theexperimental column, but 8% (74/925) to the control column.Therefore it contributes more information to the average chance of death in the experimental column than it does to the control column.There is a high chance of death in this trial, so the chance of death for the expt column is higher than the control column.
Interpretation
Interpretation - “Evidence of absence” vs “Absence of evidence”
• If the confidence interval crosses the line of no effect, this does not mean that there is no difference between the treatments
• It means we have found no statistically significant difference in the effects of the two interventions
Review: SteffComparison: 01 Absence of evidence and Evidence of absence Outcome: 01 Increasing the amount of data...
Study Treatment Control OR (fixed) OR (fixed)or sub-category n/N n/N 95% CI 95% CI
1 study 10/100 15/100 0.63 [0.27, 1.48] 2 studies 20/200 30/200 0.63 [0.34, 1.15] 3 studies 30/300 45/300 0.63 [0.38, 1.03] 4 studies 40/400 60/400 0.63 [0.41, 0.96] 5 studies 50/500 75/500 0.63 [0.43, 0.92]
0.1 0.2 0.5 1 2 5 10
Favours treatment Favours control
In the example below, as more data is included, the overall odds ratio remains the same but the confidence interval decreases.
It is not true that there is ‘no difference’ shown in the first rows of the plot – there just isn’t enough data to show a statistically significant result.
Interpretation - Weighing up benefit and harm
• When interpreting results, don’t just emphasise the positive results.
• A treatment might cure acne instantly, but kill one person in 10,000 (very important as acne
is not life threatening).
Interpretation - Quality• Rubbish studies = unbelievable results
• If all the trials in a meta-analysis were of very low quality, then you should be less certain of your conclusions.
• Instead of “Treatment X cures depression”, try “There is some evidence that Treatment X cures depression, but the data should be
interpreted with caution.”
Summary• Choose an appropriate effect measure• Collect data from trials and do a meta-analysis if appropriate• Interpret the results carefully
– Evidence of absence vs absence of evidence– Benefit and harm– Quality– Heterogeneity
Sources of statistics help and advice
Cochrane Handbook for Systematic Reviews of Interventions http://www.cochrane.org/resources/handbook/index.htm
The Cochrane distance learning materialhttp://www.cochrane-net.org/openlearning/
The Cochrane RevMan user guide.http://www.cc-ims.net/RevMan/documentation.htm