75th percentile = 2.84 25th percentile = 0.65

Median 1.39

Most Common Journals

MGH-NIH Other NIH Not NIH

PNAS (2%) J Biol Chem (3%) PLoS ONE (1%)

J Biol Chem (2%) PNAS (2%) Phys Rev Let (<1%)

PLoS ONE (2%) PLoS ONE (2%) J Biol Chem (<1%)

J Clin Endocrin Metab (2%) J Immunol (1%) BMJ (<1%)

Neuro Image (1%) Biochem (1%) Lancet (<1%)

J Neurosci (1%) J Neurosci (1%) PNAS (<1%)

J Immunol (1%) J Virol (1%) Nature (<1%)

8

“Payoffs from research follow a power law, with big, near-unlimited upside but limited downside. Since the winner will have an explosive payoff, the right approach requires a certain style of blind funding … [of] as large a number of projects as possible...” Nassim Nicholas Taleb

Some Other Thoughts …

Taleb N 2012 Random House

Another Finish Line?

10 Shapiro D, Vrana K. PNAS 2015;112:9496-7

Ability to Respond to Emergencies …

11

Science 2016;353:1094-5

Rigor, Reproducibility, and …

12

Nature 2014;505:612-13

Registration and Reporting (Fast) …

13

“To realize the benefits of a clinical trial, the data must be broadly shared quickly. The DHHS has released a regulation for registration and summary results reporting. The NIH will withhold clinical trial funding if the agency is unable to verify adequate registration and results reporting…”

JAMA 2016 (online September 16, 2016)

What We Care About – Research Output

14 Moses H et al. JAMA 2015;313:174-189

15

Ivacaftor

FDA trials

Documents / Publications

Scientists (N=2587) Institutions (N=2516)

Sanders Williams R et al. Cell 2015;163:21-23

A Metrics Story with Big Data

16

Thanks Jim Onken, Brian Haugan, George Chacko, Shixin Jiang, Samet Keserci, Alex Pico, and Lindsay Wan

PCSK9 inhibitor

Trials

Foundational publications

NIH-supported organizations NIH grants

Cited References

Why Should We Care? “Eroom’s Law”

17

Scannell JW et al. Nature Reviews Drug Discovery 2012;11:191-200

Why Might This Be Happening?

18

Scannell JW et al. Nature Reviews Drug Discovery 2012;11:191-200

Some Think Our Paradigm May be Wrong…

19

Gittelman M. J Research Policy 2016;45:1570-85 Plenge RM et al. Nature Reviews Drug Discovery 2013;12:581-594

20 http://www.nih.gov/sites/default/files/about-nih/strategic-plan-fy2016-2020-508.pdf

Towards “Evidence-Based Funding” …

21

Azoulay P. Nature 2012;484:31-32 Ioannidis J. Nature 2011; 477:529-31

“It is a scandal that billions of dollars are spent on research without knowing the best way to distribute that money.” John Ioannidis

Looking Forward to Open Dialogue …

22 http://nexus.od.nih.gov/all/category/open-mike/