Rob Davidson at the G3 Workshop: Open Source - Tools for Reproducibility
Unsw reproducibility workshop
-
Upload
alex-holcombe -
Category
Science
-
view
272 -
download
0
Transcript of Unsw reproducibility workshop
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
A new era of large-scale replication and collaboration?
@ceptional
http://www.slideshare.net/holcombea/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
False positives zone Look for areas wherein victory declared whenever null hypothesis is rejected, where size of effect, plausibility considered unimportant.
Where science is not self-correcting
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
"This one's Psych Science. 1998. Put your money on ‘will replicate’?”
"What, you think I was born yesterday?”
"Fine," muttered the bookie.
#AcademicNoir
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Well, Mr. P. Hacker, your luck is about to run out… down at the journals, they’re running a new game. They call it ‘preregistration’.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
5
https://osf.io/tvyxz/wiki/home/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
6
https://osf.io/tvyxz/wiki/home/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
7
https://osf.io/tvyxz/wiki/home/
100% P-hacking FreeHere, check our numbers.
Here’s how you can replicate our result.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
"This one's Psych Science. 1998. Put your money on ‘will replicate’?”
"What, you think I was born yesterday?”
"Fine," muttered the bookie.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 9
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 10
Failed to replicate my result?
Have you considered
the unknown moderators?
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 11
Any one of us alone, he’ll say we messed up. But if we all run the replication together, we
can show the big guy we’re right.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Meta-analysis forest plot
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Meta-analysis forest plot
Failed to replicate my result?
Have you considered the unknown moderators?
You messed up!
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
PreregistrationLarge-scale replicationOpen dataOpen protocol and code
A new era of large-scale replication
and collaboration?
Can we do this bottom-up?
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php