CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in...
-
Upload
jose-archer -
Category
Documents
-
view
214 -
download
0
Transcript of CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in...
![Page 1: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/1.jpg)
CAMARADES: Bringing evidence to translational medicine
Study Quality and Publication Bias in Experimental Stroke
![Page 2: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/2.jpg)
CAMARADES: Bringing evidence to translational medicine
Presenter Disclosure Information
FINANCIAL DISCLOSURE:None
Malcolm Macleod PhD FRCPStudy Quality and Publication Bias in Experimental Stroke
UNLABELED/UNAPPROVED USES DISCLOSURE:NoneCONFLICT OF INTEREST DISCLOSURE:President, EuroHYP
![Page 3: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/3.jpg)
CAMARADES: Bringing evidence to translational medicine
1026
1026 interventions in experimental stroke
Tested in experimentsO’Collins et al, 2006
![Page 4: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/4.jpg)
CAMARADES: Bringing evidence to translational medicine
1026603
1026 interventions in experimental stroke
Tested in focal ischaemiaO’Collins et al, 2006
![Page 5: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/5.jpg)
CAMARADES: Bringing evidence to translational medicine
1026883374
1026 interventions in experimental stroke
Effective in focal ischaemiaO’Collins et al, 2006
![Page 6: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/6.jpg)
CAMARADES: Bringing evidence to translational medicine
1026883550
97 18
1026 interventions in experimental stroke
Tested in clinical trialO’Collins et al, 2006
![Page 7: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/7.jpg)
CAMARADES: Bringing evidence to translational medicine
1026883550
97 171 3
1026 interventions in experimental stroke
Effective in clinical trialO’Collins et al, 2006
![Page 8: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/8.jpg)
CAMARADES: Bringing evidence to translational medicine
Control half dose full dose
Infa
rct V
olum
e
0
50
100
150
200
250
300
10-120 M 10-60 M
![Page 9: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/9.jpg)
CAMARADES: Bringing evidence to translational medicine
What is translational medicine?
From: Institute of Translational Medicine, University of Texas Medical Branch
![Page 10: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/10.jpg)
CAMARADES: Bringing evidence to translational medicine
Animal data in stroke
• There are huge amounts of often confusing data
• Systematic review can help to make sense of it
• If you select extreme bits of the evidence you can “prove” either harm or substantial benefit
• However, if you have a precise and highly significant overall effect, then it is probably real
Hypothermia: a systematic search identified 277 experiments in
3353 animals
Bett
er
Wors
e
Van der Worp et al, 2007
![Page 11: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/11.jpg)
CAMARADES: Bringing evidence to translational medicine
![Page 12: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/12.jpg)
CAMARADES: Bringing evidence to translational medicine
Potential sources of bias in animal studies
• Internal validity
• External validity– Publication bias – Are the models we use good models?
• Co-morbidities
Problem Solution
Selection Bias Randomisation
Performance Bias Allocation Concealment
Detection Bias Blinded outcome assessment
Attrition bias Reporting drop-outs/ ITT analysis
![Page 13: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/13.jpg)
CAMARADES: Bringing evidence to translational medicine
Internal ValidityRandomisation and blinding in studies of hypothermia in experimental stroke
• Infarct Volume– 101 publications– 222 experiments– 3256 animals– Improved outcome by
43.5% (40.1-47.0) Blinded outcome
assessment
Yes No
Effi
cacy
47%39%
Randomisation
47%37%
Effi
cacy
Van der Worp et al, 2007
![Page 14: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/14.jpg)
CAMARADES: Bringing evidence to translational medicine
Internal Validity NXY-059
• Infarct Volume– 11 publications
– 29 experiments
– 408 animals
– Improved outcome by 44% (35-53%)
Macleod et al, 2008
![Page 15: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/15.jpg)
CAMARADES: Bringing evidence to translational medicine
The File Drawer problemPublication bias
0worse better
![Page 16: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/16.jpg)
CAMARADES: Bringing evidence to translational medicine
BetterWorse
Pre
cisi
on
• All outcomes– 29 publications– 109 experiments– 1596 animals– Improved outcome by 31% (27-35%)
External ValidityPublication Bias for FK506
Macleod et al, 2005
![Page 17: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/17.jpg)
CAMARADES: Bringing evidence to translational medicine
Publication bias – G-CSF
England T et al 2009
![Page 18: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/18.jpg)
CAMARADES: Bringing evidence to translational medicine
Publication bias in experimental stroke
• Only 11/525 publications (2.2%) reported no significant treatment effects
• Trim and Fill suggested ~16% (214/1573) of experiments remain unpublished
• Best estimate of magnitude of problem – Observed efficacy 31.3% (29.7-32.8)– Adjusted efficacy 23.8% (22.2-25.5)
![Page 19: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/19.jpg)
CAMARADES: Bringing evidence to translational medicine
Publication bias
Randomisation
Co-morbidity
bias
How much efficacy is left?
![Page 20: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/20.jpg)
CAMARADES: Bringing evidence to translational medicine
There are many other sources of bias (Sacket, 1979)
• Tidying up bias• Repeated peaks bias• Significance bias• Bogus control bias• Hot stuff bias• Selective reporting bias
![Page 21: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/21.jpg)
CAMARADES: Bringing evidence to translational medicine
Current performance against key quality items
RandomisationBlinded Outcome
AssessmentSample Size calculation
Stroke 36% 29% 3%
![Page 22: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/22.jpg)
CAMARADES: Bringing evidence to translational medicine
How does stroke compare?
RandomisationBlinded Outcome
AssessmentSample Size calculation
Stroke 36% 29% 3%
MND 31% 20% <1%
AD 15% 25% 0%
PD 12% 15% 0%
EAE 8% 15% <1%
Glioma 14% 0% 0%
![Page 23: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/23.jpg)
CAMARADES: Bringing evidence to translational medicine
Internal validity in PD models
Blinded outcome assessment Composite quality
Rooke et al, submitted
![Page 24: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/24.jpg)
CAMARADES: Bringing evidence to translational medicine
Estimates of affinity at cannabinoid receptors
McPartland et al 2007
![Page 25: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/25.jpg)
CAMARADES: Bringing evidence to translational medicine
AnimalStudies
Clinical Trial
Systematic Review
AndMeta-analysis
![Page 26: CAMARADES: Bringing evidence to translational medicine Study Quality and Publication Bias in Experimental Stroke.](https://reader037.fdocuments.net/reader037/viewer/2022110115/5513d06955034646298b50ca/html5/thumbnails/26.jpg)
CAMARADES: Bringing evidence to translational medicine
Conclusions
• We could, and should, do a lot better• Other diseases could do much much better• Like any other process, the processes of
translational medicine can be studied, systematically and methodically