CSR Advisory Council Meeting May 19, 2014 Editorial Board Review A Few Good Reviewers Don Schneider,...
13
CSR Advisory Council Meeting May 19, 2014 Editorial Board Review A Few Good Reviewers Don Schneider, Ph.D.
-
Upload
bethanie-powell -
Category
Documents
-
view
216 -
download
0
Transcript of CSR Advisory Council Meeting May 19, 2014 Editorial Board Review A Few Good Reviewers Don Schneider,...
- Slide 1
- CSR Advisory Council Meeting May 19, 2014 Editorial Board Review A Few Good Reviewers Don Schneider, Ph.D.
- Slide 2
- Reviewer pools Past experience Cost considerations Toward Review by the Best
- Slide 3
- Reviewer StatusNumbers HHMI300 NIH R37700 NAS2,000 R0126,000 CSR needs16,000 Reviewer Pools
- Slide 4
- Format modeled on journal manuscript review First piloted in 2008 with 6 SBIR panels, just in time for TR01s, Challenge grants, DP1 etc. Two stages First Stage Mail reviewers Second Stage Editors Past Experience
- Slide 5
- First Stage/Mail Reviewers Subject matter experts Provide depth in review Focus on scientific and technical merit 2-3 first stage mail reviewers per application Submit full critiques Give overall impact and criterion scores Overall impact scores not factored into final priority score
- Slide 6
- Hold face-to-face meeting Recruit broad experts Provide perspective in review (assign about 15 applications each) Focus on impact and significance Assign 3 second stage reviewers per application Consider first stage critiques in review Write overall impact paragraph Give overall impact score Final priority score based on second stage only Second Stage/Editors
- Slide 7
- Provides both depth and breadth in review Optimizes use of the best reviewers Scales well for large numbers of applications (second stage discusses a fraction of the applications) Rationale
- Slide 8
- Perceived Advantages Involves no travel/teleconference for first stage reviewers Allows small, interactive face-to-face meetings Promotes better scoring and assessment of impact Lessens travel and lodging expenses and inconveniences
- Slide 9
- Review# of ApplicationsCost/application Regular R01 F2F$518 DP1/Pioneer EB+I244$280 DP2/New Innov EB593$124 DP5/Early Indep EB+I84$875 Cost Considerations (Alicia Caffi)
- Slide 10
- Recruitment of large numbers of reviewers Timeline Tight, two sequential reviews (in the 17 week cycle) More staff time required (SROs) Some sense of isolation by first stage reviewers Challenges
- Slide 11
- Each application examined by at least 5 reviewers Interactive, thoughtful discussions Overall scoring by second stage members Reviewers and staff like final review products Review Outcomes
- Slide 12
- Survey conducted by A Kopstein of reviewers participating in SBIR pilots 2008 Outcomes were generally positive Majority willing to participate in either review stage in future Editorial Board Review: Increases expert review 3/4 ths of respondents Preferred for their own applications 2/3 rds of respondents Survey
- Slide 13
- Hopes for a few good reviewers? Discussion