Meta-Analyses: Appropriate Growth or Malignant Tumor?

• Moderator– Alan M. Batterham, FACSM

University of Teesside, Middlesbrough, United Kingdom.

• Speakers– Will G. Hopkins, FACSM

AUT University, Auckland, New Zealand.– Ian Shrier, FACSM

McGill University, Montreal, PQ, Canada.

Historical Notes

• Karl Pearson (1857-1936)– ‘‘Many of the groups … are

far too small to allow of any definite opinion being formed at all, having regard to the size of the probable error involved’’

(Report on certain enteric fever inoculation statistics. BMJ, 3, 1243-6, 1904)

Historical Notes• Gene Glass the first to coin the term

‘Meta Analysis’ in 1976 (Primary, secondary, and meta-analysis of research. Educat Res, 5, 3-8)

• 1979 – Archie Cochrane (1909-1988) highlighted that healthcare decision makers had no ready access to reliable evidence reviews.

• Cochrane Collaboration launched 1993

DEFINITIONS• Systematic ReviewSystematic Review

– A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review.

• Meta-AnalysisMeta-Analysis – The use of statistical techniques (quantitative

inference) in a systematic review to integrate the results of included studies. Sometimes misused as a synonym for systematic reviews, where the review includes a meta-analysis.

DEFINITIONS

• Non-Systematic ReviewNon-Systematic Review: : Non-systematic Non-systematic selection of studies, no statistical combination selection of studies, no statistical combination - may be biased.- may be biased.– ‘‘ ‘‘Take a simmering topic, extract the juice of an Take a simmering topic, extract the juice of an

argument, add the essence of one filing cabinet, argument, add the essence of one filing cabinet, sprinkle liberally with your own publications and sprinkle liberally with your own publications and sift out the work of noted detractors or sift out the work of noted detractors or adversaries’’ (Anon.)adversaries’’ (Anon.)

– An opinion pieceAn opinion piece

REVIEWSREVIEWS

SYST.

REVIEWS

SYST.

REVIEWSCochrane-

defined meta-analysis

Cochrane-defined

meta-analysis

BIASED ESTIMATES

BIASED ESTIMATES

META ANALYSES

META ANALYSES

COCHRANE REVIEWS

0

200

400

600

800

1000

1200

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Year

# P

ub

lish

ed

in Y

ea

r

ProtocolReviews

Number of articles with ‘Meta Analysis’ in the title: ISI Web of Science

Number of articles with ‘Meta Analysis’ in the title: ISI Web of Science

0200400600800

1000120014001600

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Year of Publication

Nu

mb

er

of

art

icle

s

Fig 1 The Cochrane symbol, seen here in plans for the façade of the Cochranite Vatican that is to be built in Summertown

Fig 1 The Cochrane symbol, seen here in plans for the façade of the Cochranite Vatican that is to be built in Summertown

Clinicians for the Restoration of Autonomous Practice Writing Group (BMJ, 325, 1496-1498, 2002)Clinicians for the Restoration of Autonomous Practice Writing Group (BMJ, 325, 1496-1498, 2002)

EBM factions (BMJ, 325, 1496-1498)

• Cochranites - members of a worldwide order that has Archie Cochrane as its patron saint. They worship systematic reviews

• Meta anophiles - a heretical spin off of the Cochranites, who believe in combining the results of all the randomised controlled trials in the universe. Their current estimate of the ultimate summary statistic is 42 (95% CI 41.9999 to 42.0001)