1

Trends and Updates: Research on Research Integrity

Trends and Updates: Research on Research Integrity

Nicholas H. Steneck, PhD

University of Michigan2009 ORI RRI Conference:

THE CONFERENCE CENTER NIAGARA FALLS, NY

May 15-17, 2009

2

Origins of RRI at ORI

1999 Chris Pascal / Larry Rhoades request $1M for RRI

• HHS (NIH source of funds) approves• Mary Scheetz, ORI program officer

• now Cynthia Ricard <Cynthia.Ricard@hhs.gov>

• Contract to study of RRI literature• Contacted researchers for advice

July 1999, contact N. Steneck regarding four projects:• November advisory committee meeting• National RRI Conference• RRI program• RRI Literature review

3

ORI justification (Pascal 2000)

Since its establishment in 1992, the Office of Research Integrity (ORI) has conducted several studies on research misconduct and research integrity in an attempt to develop a knowledge base on important issues, such as the impact of misconduct allegations on exonerated scientists, the experience of whistleblowers in the aftermath of making allegations, the research guidelines adopted by medical schools, and the causes of research misconduct.

Over time, it became apparent to ORI that a more comprehensive, coordinated effort in collaboration with extramural research scholars was needed to develop the science base on research integrity issues. This recognition led to development of this "Research Conference on Research Integrity" and the related "Research on Research Integrity" program announcement jointly issued by ORI and the National Institute of Neurological Disorders and Stroke (RFA: NS-01-008).

4

Planning Committee

Ruth Fischbach, Ph.D., M.P.E., National Institutes of Health Mark Frankel, Ph.D., American Association for the Advancement of Science Paul Friedman, M.D., UC, San Diego Edward Hackett, Ph.D., Arizona State University Stanley Korenman, University of California, San Diego Francis Macrina, Ph.D., Virginia Commonwealth University Barry Markovsky, Ph.D. University of Iowa Kathleen Montgomery, Ph.D., UC, Riverside John Perhonis, Ph.D., National Science Foundation Drummond Rennie, M.D., F.R.C.P., F.A.C.P., UC, San Francisco Allan Shipp, M.H.A., Association of American Medical Colleges Peter Yeager, Ph.D., Boston University

5

Major events RRI (& RCR) Conferences

November 2000, Bethesda MD (Marriott) November 2002, Potomac MD (Bolger Conference Center) November 2004, San Diego CA (Paradise Point Resort)

• UC San Diego School of Medicine December 2006, Tampa FL (Safety Harbor Resort)

• University of South Florida College of Medicine April 2007, St. Louis MO (Washington University)

• Washington University in St. Louis School of Medicine May 2009, Niagara Falls NY (Conference Center)

• Roswell Park Cancer Institute

RRI Program, December 15, 2000, deadline, 1st round ORI does not have granting authority Early 2000 explored administration options

6

May NIH presentation, IC Directors

7

Closing slides

NINDS agreed to be 1st co-sponsor

8

RRI Program HistoryAREA / Year 2001 2002 2003 2004 2005 2006 2007 2008 Total %

Education 5 7 9 10 5 3 4 6 49 18%Conflict of Interest 7 2 2 6 4 5 1 0 27 10%

Misconduct 3 4 2 4 1 0 1 1 16 6%Social Aspects 4 6 9 14 18 9 9 9 78 29%

Human subjects 5 5 5 11 9 18 3 3 59 22%Data management 0 1 2 3 3 3 3 1 16 6%

Publication 1 4 3 4 5 3 1 3 24 9%

Total received 25 29 32 52 45 41 22 23 269 100%

# Funded 7 9 6 5 7 5 7 3 49

% Funded 28% 31% 19% 10% 16% 12% 32% 13% 18%

NINDS NINR NCRR

2001, RO1, two-year, $100,000/year, direct cost 2004, three-year, $250,000/year direct cost

Most proposals received Lowest percentage of proposal funded

2005, two-year, $175,00

2008, R22

9

Funding rate average = 18%

28%31%

19% 10% 16% 12% 32%13%

10

Source of funds / totals

NIH provides over 50%

11

ORI / Other funding (60% / 40%)

12

Primary focus, all proposals

13

Areas: Proposed / Funded

14

RRI apart from ORI

NIH funds training and research ethics, not RRI Ethics = bioethics, not research ethics/integrity RCR funded through training grants RRI can be considered but few applications

NSF broad array of programs STS –> EVS, HPS, SSS, and SPS Research ethics training (REU, IGERT, etc.)

Peer Review Congresses Research on peer review and biomedical publication Peer generated and supported; no funding

15

NSF vs. NIH, RCR education

ORI/NIH RCR Enhancement Initiative (2002) Budget

• National planning and coordination effort ($1.1m),• Institutional Support Program ($11.5m), and• Instructional Enhancement Program ($6.1m).

Approved by NIH Director in 2004, never implemented

National Science Foundation, research ethics 2001-2009, 328 “research ethics” awards

• $270,616,429.00 total funding• Most to support instructional development

No special attention to or coordination of RRI

*

ORI has provided some RCR support

16

Global interest in RRI?

Some interest in Europe: Early pioneer, Croatia, now joined by other countries European Science Foundation has RRI committee

• Just getting organized

UK RIO has considered, but not yet undertaken No support from European Commission

China is supporting some RRI Developing an instrument to assess integrity Has investigated plagiarism

Elsewhere, support for RRI is scattered

17

Observations: Professional development RRI has become a recognized field of research

Established researchers anxious to pursue Peers have reviewed and endorsed proposals Funding agencies provide some support Literature and professional community has emerged

Influences on professional development of RRI FUNDING ~ interest grows/declines with support Funding mechanism ~ RO1 vs. R22 External events

• Human subjects concerns• Conflict of interest

18

Observations: Tensions

Researcher-funder conflict Researchers interested in broad (basic) studies Funders interested in IC-specific (applied) topics

Research-policymaker conflict Researchers interested in broader (basic) studies Policy makers want studies that are relevant to policy

making Professional apathy / hostility

RRI is “important” but low priority … RRI is conspiracy to undermine science

19

RRI track record (ORI & other) Misconduct “worldview” 1970s

Serious misconduct is rare Self-regulation keeps in check Misconduct is difficult to detect Misconduct cannot be prevented Apart from misconduct, standards are high

RRI has cast doubt on all five assumptions FFP ~ 0.1–>1.0%; QRP ~ 5% –> 50% Self-regulation has serious shortcomings Better self-regulation would prevent FFP / QRP Research integrity can be improved (considerably)

20

Challenge ~ demonstrating relevance 2006 Study (NEJM)

Early detection study CT scan vs. X-ray With CT scan, 80% of cases

could be cured March 28, 2008 NY Times*

Did not report study partly funded by tobacco company

• Vector Group (Liggett) —>$$$—> Foundation for Lung Cancer: • Foundation listed in NEJM article, along with 31 additional funding sources

Did not report held patents on CT-scan-related technology• Impact/profit: 48M former smokers / 40M current smokers (US)

Others involved as members of Foundation for … Early Detection• Dean & Vice-Chair Board of Overseers, Weill Cornell Medical College

Controversial cases get attention

*see also Paul Goldberg, Cancer Letter Inc. (2008)

21

Comparison to careful RRI study

Gorman, Evaluation & Program Planning (2007) study of National Registry of Effective & Promising Programs

for drug and alcohol use prevention

• 78% of published evaluations had developer as an author

• Developers had a significant financial interest in outcomes

• Evaluation in one program had serious methodological flaws

Texas spent $35M on one questioned program in 2007

Why is this story not national headline news? COI –> wasted funds and probably loss of life

Conclusion: future of RRI depends on relevance Need more studies of impacts (economic / human)

RRI community must become proactive

22