Handout H6 The scope of meta-analysis: Meta-analysis of observational studies
description
Transcript of Handout H6 The scope of meta-analysis: Meta-analysis of observational studies
11 August 2010
Handout H6
The scope of meta-analysis: Meta-analysis of observational studies
Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Objectives
• Understand the importance of systematic reviews of observational studies
• Understand the limitations of meta-analysis in observational studies
• Understand the difficulties of avoiding publication bias in observational studies
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Why do we need observational studies?
• Randomisation may be
– impossible
– unnecessary
– inappropriateBlack, BMJ 1996
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Potentials of systematic reviews• More objective appraisal of the evidence than traditional
narrative reviews applies equally to OS & RCT
• May resolve uncertainty when original research, reviews and editorials disagree applies equally to OS & RCT
• May generate promising research questions to be addressed in future studies applies equally to OS & RCT
• Meta-analysis will enhance the precision of effect estimates, leading to reduced probability of false negative results BUT in OS may be a precise biased result
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Meta-analysis
• A statistical analysis which combines the
results of several independent studies
considered by the analyst to be
‘combinable’
Huque 1988
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Assumptions in meta-analysis
• “Fixed-effects model”: Underlying effect is the same value (fixed) in each study. The differences between study results are solely due to the play of chance.
• “Random-effects model”: Treatment effect for the individual studies are assumed to vary around some overall central effect
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Fundamental difference in assumptions & how they apply to MA of RCT or observational studies
• In meta-analysis of observational studies confounding, residual confounding and bias:
– May introduce heterogeneity
– May lead to misleading (albeit very precise) estimates
• In well-conducted RCT there should not be confounding
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Relative risk
(95% confidence interval)
0.1 0.2 0.5 1 2 5 10
Trial (Year)
Barber (1967) Reynolds (1972)
Wilhelmsson (1974) Ahlmark (1974)
Multicentre International (1975) Yusuf (1979)
Andersen (1979)
Rehnqvist (1980) Baber (1980)
Wilcox Atenolol (1980)
Wilcox Propanolol (1980) Hjalmarson (1981)
Norwegian Multicentre (1981)
Hansteen (1982) Julian (1982) BHAT (1982) Taylor (1982)
Manger Cats (1983)
Rehnqvist (1983) Australian-Swedish (1983)
Mazur (1984) EIS (1984)
Salathia (1985)
Roque (1987) LIT 91987)
Kaul (1988) Boissel (1990)
Schwartz low risk (1992)
Schwartz high risk (1992) SSSD (1993)
Darasz (1995) Basu (1997)
Aronow (1997)
Overall (95% CI) 0.80 (0.74 - 0.86)
Mortality results from 33
trials of beta-blockers in
secondary prevention after
myocardial infarction
Adapted from Freemantle et al BMJ 1999
0.2 0.5 1 2 5 10
Study
Allen Barongo Bollinger Bwayo Bwayo Cameron Carael Chao Chiasson Diallo Greenblatt Grosskurth Hira Hunter Konde-Luc Kreiss Malamba Mehendal Moss Nasio Pepin Quigley Sassan Sedlin Seed Simonsen Tyndall Urassa 1 Urassa 2 Urassa 3 Urassa 4 Urassa 5 Van de Perre
Relative risk
(95% confidence interval)
Results from 29 studies examining the association between intact foreskin
and the risk of HIV infection in men
Adapted from Van Howe Int J STD AIDS 1999
Formaldehyde exposure and lung cancer
0
50
100
150
IndustrialWorkers(14 Cohorts)
Anatomists,Pathologists(3 Cohorts)
Funeral DirectorsEmbalmers(7 Cohorts)
SM
R (
95
% C
I)
Blair et al Scan J Work Environ Health 1990
Dietary fat and breast cancer
0.6
0.8
1.0
1.2
1.4
1.6
1.8
12 Case-Control Studies
6 Cohort Studies
Rel
ativ
e R
isk
(95%
CI)
Boyd et al Br J Cancer 1993
Intermittent sunlight exposure and melanoma
0
1
2
3
7 Case-ControlStudies withBlinding
Od
ds
Rat
io (
95%
CI)
9 Case-ControlStudies withoutBlinding
Nelemans et al J Clin Epidemiol 1995
Test of homogeneity
• Examines the possibility of excess variability
between the results of the different studies
• Has low power if the number of studies is
small
• Can get a set of homogeneous but spurious
findings
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Beta-carotene and cardiovascular mortality
0.1 0.5 0.75 1 1.25 1.5
Cohort Country
Male health workers
Social insurance, men
Male chemical workers
Hyperlipidaemic men
Nursing home residents
USA
Finland
Finland
Switzerland
USA
USA
Social insurance, women
Cohorts combined
Relative risk (95% CI)
Jah et al Ann Intern Med 1995
0.1 0.5 0.75 1 1.25 1.5 1.75
Male health workers
Social insurance, men
Male chemical workers
Hyperlipidaemic men
Nursing home residents
Social insurance, women
Male physicians
Male smokers
(Ex)-smokers, asbestos workers
Trials
Cohorts
Skin cancer patients
USA
Finland
Switzerland
USA
USA
Finland
Cohorts combined
Trials combined
Finland
USA
USA
USA
Relative risk (95% CI)
Beta carotene and cardiovascular disease
Egger et al. BMJ 1998;316:140-4
“Well, so much for antioxidants.”
Smoking and suicide
0.2 1 2 5 10 25
No of cigarettes
MRFIT screenees
Whitehall I
North Karelia men
Kuopio men
1-1414-24 25+
1-1414-24 25+
1-1414-24 25+
1-1414-24 25+
1-1414-24 25+
Meta-analysis
Relative rate (95% CI)
Davey Smith et al Lancet 1992
Smoking and homicide
• Non-smoker 1.00
• 1-2 packs/day 1.71 (1.29-2.28)
• 2+ packs/day 2.04 (1.32-3.15)
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Fundamental difference in assumptions
• In meta-analysis of observational studies confounding, residual confounding and bias:
– May introduce heterogeneity
– May lead to misleading (albeit very precise) estimates
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
What is the appropriate weighting factor?
Inverse of variance?
Case-control studies of Helicobacter pylori infection and CHD
Study Cases +ve
Controls +ve
Cases -ve
Controls -ve
Crude OR
Adjusted OR
Weight
Danesh 1999a
472 272 650 850 2.3 1.9 44%
Danesh1999b
134 294 112 348 1.4 1.3 17%
Patel 1995
56 135 27 170 2.6 2.8 6%
Murray1995
102 1117 33 863 2.4 1.5 9%
McDonagh
1997
315 625 134 353 1.3 0.9 m1.0 f
25%
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Danesh et al 1999 a
Danesh et al 1999 b
Numbers in study 2224 888
Response rate controls <20% 60%
Response rate cases ~60% 56%
Adjustment for social position
+ +++
Other adjustments + +++
Representative cases? No Fairly representative
Representative controls? No Fairly representative
OR sex/age adjusted 2.3 1.4
OR fully adjusted 1.9 1.3
Weight in meta-analysis 44% 17%
Two case-control studies of Helicobacter pylori infection and CHD
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Study Cases +ve
Controls +ve
Cases
-ve
Controls -ve
Crude OR
Adjusted OR
Weight
Strachan 1998
204 1061 82 449 1.05 1.02 17%
Wald 1997 308 595 340 701 1.07 1.06 37%
Aromaa 1998 229 411 47 116 1.38 - 9%
Folsom 1998 111 257 106 241 0.98 0.97 13%
Ossewaarde 1998
39 84 15 24 0.74 - 2%
Whincup 2000
401 740 104 285 1.48 1.30 20%
Prospective and nested case-control studies of Helicobacter pylori infection and CHD
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
In RCT meta-analyses the appropriate study weights should relate to precision of effect estimates (e.g. inverse of variance).
In observational meta-analyses this may not generally be the case.
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Inverse of variance weighting
• Can lead to magnification of pooled effect estimates when confounding and bias involved (e.g. H pylori)
• Can lead to under-estimation of effect estimates when measurement error is important
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Huxley, Lancet 2002
Publication bias in MA / SR of observational studies
Data analysed
Paper published
Reporting bias in observational research
Data collected(e.g cohort study)
Report written
“I regret to inform you that the Journal of xxx will not be able to use your manuscript … We think the study is well-designed, with a fair follow-up and appropriate statistical
analysis, but the negative results found can only be published as a Letter to the editor …”
Rejection of ‘negative’ prospective cohort study finding of association of d-dimer with CHD. May 2006
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Genetic meta-analysis may be an exception …
To the homogeneity and “spurious precision” problems…
But may be particularly prone to publication bias
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
0.33 0.5 0.66 1.0 1.5 2.0 3.00.33 0.5 0.66 1.0 1.5 2.0 3.0
0.0
0.1
0.2
0.3
0.4
Odds ratio
Sta
nd
ard
err
or
Funnel plot of meta–analysis of ACE I/D and CHD
Conclusions• The principles of systematic reviews are applicable to
any research design
• Reviews of observational studies should always be systematic
• Much attention should be given to exploring possible sources of heterogeneity
• HOWEVER: Meta-analysis of observational studies will often produce misleading and spuriously precise estimates
• Trial registers should solve much of the problems of publication bias in RCT, but trying to solve publication bias in observational studies impossible?
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme
Future work
• We need to define optimal search strategies to identify epidemiological studies in the literature
• We need validated instruments to assess the study quality at the design, conduct and analysis level
• We need to improve the quality of reporting of epidemiological studies
• We need to facilitate individual patient data analyses
• We need to better define the place of meta-analysis in systematic reviews of epidemiological studies
11 August 2010 Topics in Meta-Analysis (Matthias Egger) Erasmus Summer Programme