Ability of New Innovator Awardees to Obtain Subsequent NIH ... · more successful than the other...
Transcript of Ability of New Innovator Awardees to Obtain Subsequent NIH ... · more successful than the other...
Office of Portfolio Analysis (OPA) Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI) Office of the Director (OD)
CASE STUDY
Ability of New Innovator Awardees to Obtain Subsequent NIH Support
January 2015
2
Table of Contents Background ................................................................................................................................................... 3
Methods ........................................................................................................................................................ 4
Identification of the cohorts of interest ................................................................................................... 4
STEP 1: DEFINE YOUR DATASET ............................................................................................................ 4
STEP 2: SEARCHING FOR GRANTS USING iSearch ................................................................................. 4
STEP 3: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) ....................................................... 4
Subsequent awards ................................................................................................................................... 4
STEP 4: SEARCHING FOR SUBSEQUENT AWARD DATA USING iSearch ................................................. 5
STEP 5: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) ....................................................... 5
Results ........................................................................................................................................................... 6
DP2, R00, and R01-ESI cohorts.................................................................................................................. 6
DP2, R00, and R01-ESI awardees with subsequent NIH awards ............................................................... 7
Success rate of awarded applications ....................................................................................................... 8
Percentage of multi-PI applications .......................................................................................................... 9
NIH-awarded applications by activity code ............................................................................................ 10
Length of time to first subsequent NIH award ....................................................................................... 11
Conclusions ................................................................................................................................................. 12
Background
Between 1980 and 2001, the average age at which principal investigators (PIs) receive their first R01
award increased from approximately 36 to 42, where it has remained stable since. In 2009, NIH
established the Early Stage Investigator (ESI) status for PIs who are within 10 years of receiving their
terminal degree and have not yet received NIH funding as a PI. The purpose of this policy change was to
encourage peer reviewers and ICs to take into account the career stage of investigators when evaluating
their projects. Though ESI status is primarily intended to assist new investigators applying for research
support via traditional funding mechanisms such as R01 awards, several NIH mechanisms are specifically
directed at ESIs, including the NIH Director’s New Innovator Award (DP2).
The NIH DP2 program, which began in FY2007, is intended to support investigators of exceptional
creativity with the potential to produce high impact research. Only ESIs are eligible for DP2 awards. DP2
awardees receive five years of support, and lose their ESI status when submitting subsequent
applications. Peer review of DP2 applications is distinct from the traditional NIH study section process,
and attempts to identify the most creative applicants and projects with transformative potential in the
applicant’s field of biomedical/behavioral research.
Another NIH mechanism directed at ESIs is the Pathway to Independence Award (K99/R00). The goal of
the K99/R00 program, which began in FY2006 and targets researchers with no more than four years of
postdoctoral experience, is to increase and maintain a strong cohort of new and talented independent
investigators. K99/R00 awardees receive up to five years of support, consisting of two phases: an initial
mentored research interval (K99 phase, approximately two years), followed by the scientist's first
independent research support phase (R00 phase, approximately three years). K99/R00 recipients
maintain their ESI status when applying for another NIH award.
The purpose of this analysis is to investigate how successful DP2 awardees are in obtaining subsequent
NIH awards, and compare them with two control groups: R00 awardees and a randomly selected cohort
of R01-ESI awardees.
4
Methods
Identification of the cohorts of interest We extracted the data for all Type 1 DP2, R00, and R01-ESI awardees (FY2009 through FY2012) from
iSearch.
STEP 1: DEFINE YOUR DATASET The cohorts of interest for this study are all DP2, R00, and R01-ES1 awardees from FY2009-2012.
STEP 2: SEARCHING FOR GRANTS USING iSearch If you are not currently an iSearch user, please send an e-mail to [email protected] to request access.
Field DP2 R00 R01
ADVANCED FILTER
Fiscal Year range 2009 - 2012 2009 - 2012 2009 - 2012
Awarded
Yes (+) Yes (+) Yes (+)
Activity Code
DP2 R00 R01
FACET TAB
Agency
NIH NIH NIH
Application Type 1 4 1
Number of awardees
206 692 3009
STEP 3: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) To download the results for each search hit the Download button in the top right of the screen. Select ‘Export to CSV (20k limit)’, and select Contact PI as your download field. You will have three csv files that you will use in step 4.
In order to test a comparable number of R01-ESI awardees against the DP2 and R00 awardees, 100 R01-
ESI awardees were selected at random for each fiscal year (400 awardees total), and this cohort was
used for identification of subsequent awards.
Subsequent awards Subsequent applications and awards for these cohorts were identified as follows:
All Type 1 (T1) applications
All P, R, and/or U mechanisms
5
The time interval ranged from one year after the initial DP2/R00/R01-ESI award through the end
of FY 2014. For example, if an investigator received a DP2 award in FY2009, the interval in which
we searched for subsequent applications and awards was FY2010 through FY2014.
STEP 4: SEARCHING FOR SUBSEQUENT AWARD DATA USING iSearch
1. For each ESI cohort, prepare a CSV file with a single column containing the Contact PI names
(delete the header and first column from your iSearch download).
2. Making sure that you have cleared your last search, upload your files, one at a time by clicking
on the Upload button and browsing to where you saved your files. Select one of the CSV files
you created, and click Upload. This will automatically run the search and return all applications
for your cohort of interest.
3. Limit the results by using the Filter option with the following search criteria:
Fiscal Year: 2010 - 2014
Agency: National Institutes of Health
4. On the facet view select a. Type 1 applications. b. P, R, and U mechanisms:
i. click on the Menu icon in the top right corner of the Activity Code facet ii. Type P, R, or U into the search bar and select all activity codes of interest by
holding Ctrl while clicking. iii. Drag the selected activity codes into the Selected Facets box and ensure that
the radio button for delimiting the search is set to OR:
Click Apply Changes.
Subsequent awards data was downloaded and used to determine various aspects of awardee success.
STEP 5: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA)
To download the results, select the Results tab then hit the Download button . Select ‘Export to Excel (20k limit)’. You can reduce the number of fields in the download if you wish but include Contact PI, ApplID, Multi PI Indicator, Activity Code, Award Status, and Fiscal Year at minimum.
6
Results
DP2, R00, and R01-ESI cohorts From FY2009 to FY2012 (Table 1) there were:
206 DP2 awardees
692 R00 awardees
3,037 R01-ESI awardees
o 400 R01-ESI awardees (100 per year) were randomly selected as a control group
Table 1. Number of awardees in each cohort.
7
DP2, R00, and R01-ESI awardees with subsequent NIH awards Over half of DP2 and R00 awardees who submitted a subsequent T1 award were successful (61.8% and
54.0%, respectively), compared with 37.7% of the R01-ESI cohort (Figure 1). DP2 awardees were also
more successful than the other two cohorts in obtaining more than one subsequent T1 award by the
end of FY2014 (24.1%, 15.0%, and 14.4% of DP2, R00, and R01-ESI awardees, respectively).
Figure 1. (A) Subsequent T1 applications from awardees in each cohort. (B) Awardees in each cohort
with one or more subsequent T1 awards.
0
20
40
60
80
DP2 R00 R01-ESI
% w
ith
su
bse
qu
ent
awar
d
% w/only one award % w/more than one award
A B
8
Success rate of awarded applications We also determined the success rate for each cohort by calculating the number of awards as a
percentage of the total number of T1 applications (Figure 2). The success rate was similar for the DP2
and R00 cohorts (20.1% and 19.4%, respectively), and lower for the R01-ESI cohort (16.5%). The
maximum number of applications per PI was similar across all three cohorts, as was the maximum
number of awarded applications (Figure 2A).
Figure 2. (A) Fate of each cohort’s subsequent applications. (B) Success rate for each cohort.
0
5
10
15
20
25
DP2 R00 R01-ESISu
cce
ss r
ate
(%)
A B
9
Percentage of multi-PI applications Some new investigators might attempt to increase their chance of getting an award by applying for a
multi-PI award, often partnering with more established investigators. The total number of multi-PI
applications was determined for each cohort. The percentage of all applications that were multi-PI was
higher for DP2 awardees than for R00 or R01-ESI awardees, as was the percentage of those multi-PI
applications that received an award (Figure 3).
Figure 3. (A) Multi-PI applications and awards. (B) Percentage of multi-PI applications and awards.
0
25
50
75
100
DP2 R00 R01-ESIFu
nd
ed a
pp
licat
ion
s (%
)% P awards % R01s awards
% other R awards % U awards
A B
Table 4. Multi-PI applications and awards.
10
NIH-awarded applications by activity code Most subsequent awards in each cohort were R01s, followed by other R awards (e.g., R21s) and a small
number of P and U awards (Figure 4). Note that a similar percentage of subsequent R01 awards to DP2
and R01-ESI investigators were multi-PI (29.9% and 32.5%, respectively); for R00 awardees, a much
lower percentage of subsequent R01 awards were multi-PI (8.2%).
Figure 4. (A) Funded applications by activity code. (B) Percentage of funded applications by activity code.
0
25
50
75
100
DP2 R00 R01-ESI
Fun
ded
ap
plic
atio
ns
(%)
% P awards % R01s awards
% other R awards % U awards
A B
11
Length of time to first subsequent NIH award Among those who submitted at least one application (see Figure 1A), 40.6% of DP2 awardees obtained a
subsequent award within two years, as opposed to 23.5% and 19.9% of R00 and R01-ESI awardees,
respectively (Table 2, Figure 5). The most striking difference was observed within the first year, when
over a quarter of DP2 PIs had obtained a subsequent award
Table 2. Length of time to first subsequent NIH T1 award.
Figure 5. Percentage of awardees in each category receiving a T1 award in subsequent years.
12
Conclusions
Of those submitting subsequent applications, DP2 and R00 awardees were more successful than the
R01-ESI control group (61.8%, 54.0%, and 37.7% of applicants, respectively, received an award; Figure
1). DP2 and R00 awardees were also slightly more successful than R01-ESI awardees on a per application
basis, calculated as the percentage of all T1 applications that received an award (20.1%, 19.4%, and
16.5% respectively; Figure 2). The percentage of awardees applying for subsequent awards, as well as
the average and maximum number of applications per PI, were similar among all three cohorts (Figures
1A and 2A).
A higher percentage of DP2 awardees applied for funding as part of a multi-PI application relative to the
other two cohorts, and DP2 awardees were also more successful at obtaining multi-PI funding (Table 4;
Figure 3). The majority of the awards in each cohort were R01s, followed by other R mechanisms; U and
P awards comprised a small percentage of awards in all three cohorts (Figure 4).
Overall, DP2 awardees seem to be more successful than the other two cohorts in obtaining subsequent
NIH funding. Relative to R00 and R01-ESI PIs, more than twice the percentage of DP2 PIs who applied for
an award were successful within one year (8.9%, 11.3%, and 25.9%, respectively). The method of peer
review used to evaluate DP2 applications, which is distinct from the traditional NIH method of peer
review, therefore appears to succeed in identifying more successful grant-writers.