An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director,...

31
An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association for Thoracic Surgery March 4, 2011 National Institutes of Health U.S. Department of Health and Human Services

TAGS:

Transcript of An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director,...

Page 1: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

An Overview of Peer Review at CSR – Critical Do’s and Don’ts

Joy Gibson, D.Sc. Director,

Division of Translational and Clinical Sciences

American Association

for Thoracic Surgery

March 4, 2011

National Institutes of HealthU.S. Department of Health and Human Services

Page 2: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

National Institutes of HealthNational Institutes of Health

National Instituteon Alcohol Abuseand Alcoholism

National Instituteon Alcohol Abuseand Alcoholism

National Instituteof Arthritis andMusculoskeletal

and Skin Diseases

National Instituteof Arthritis andMusculoskeletal

and Skin Diseases

National CancerInstitute

National CancerInstitute

National Instituteon Drug Abuse

National Instituteon Drug Abuse

National Instituteof Environmental Health Sciences

National Instituteof Environmental Health Sciences

National Instituteon Aging

National Instituteon Aging

National Instituteof Child Health

and HumanDevelopment

National Instituteof Child Health

and HumanDevelopment

National Institute onDeafness and Other

CommunicationDisorders

National Institute onDeafness and Other

CommunicationDisorders

National EyeInstitute

National EyeInstitute

National HumanGenome Research

Institute

National HumanGenome Research

Institute

National Instituteof Mental Health

National Instituteof Mental Health

National Instituteof NeurologicalDisorders and

Stroke

National Instituteof NeurologicalDisorders and

Stroke

National Instituteof General

Medical Sciences

National Instituteof General

Medical Sciences

National Instituteof Nursing Research

National Instituteof Nursing Research

National Libraryof Medicine

National Libraryof Medicine

Center for Center for Scientific ReviewScientific Review

National Centerfor Complementary

and AlternativeMedicine

National Centerfor Complementary

and AlternativeMedicine

National Instituteof Allergy and

Infectious Diseases

National Instituteof Allergy and

Infectious Diseases

FogartyInternational

Center

FogartyInternational

Center

National Centerfor ResearchResources

National Centerfor ResearchResources

Clinical Center

Clinical Center

National Center on Minority Health andHealth Disparities

National Center on Minority Health andHealth Disparities

National Institute of Biomedical Imagingand Bioengineering

National Institute of Biomedical Imagingand Bioengineering

Office of the DirectorOffice of the Director

Center for InformationTechnology

Center for InformationTechnology

National Heart,Lung, and Blood

Institute

National Heart,Lung, and Blood

Institute

National Instituteof Dental andCraniofacial

Research

National Instituteof Dental andCraniofacial

Research

National Instituteof Diabetes andDigestive and

Kidney Diseases

National Instituteof Diabetes andDigestive and

Kidney Diseases

Page 3: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

National Institutes of Health

Center for Scientific ReviewCenter for Scientific Review

Study SectionStudy Section

Institute

Advisory Councils and Boards

Institute Director

Assigns to IC & I IRG/Study Section

Reviews for Scientific Merit

Evaluates for Relevance

Recommends Action

Takes Final Action

InitiatesResearch Idea

ConductsResearch

Allocates FundsAllocates Funds

Submits ApplicationSubmits Application

Review Process for a Research GrantReview Process for a Research Grant

Research Grant Application

School or Research Center

Page 4: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Scientific Review ProcessScientific Review ProcessDual Review System for Grant ApplicationsDual Review System for Grant Applications

Second Level of ReviewSecond Level of Review NIH Institute/Center NIH Institute/Center

CouncilCouncil

First Level of ReviewFirst Level of Review

CSR or Institute ReviewCSR or Institute Review

Scientific Review Group Scientific Review Group (Study Section)(Study Section)

Page 5: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

• Receives all NIH applications

• Refers them to NIH Institutes/Centers and to scientific review groups

• Reviews majority of grant applications for scientific merit

Your Application Goes to the Your Application Goes to the NIH Center for Scientific Review (CSR)NIH Center for Scientific Review (CSR)

Focal Point for Initial Review at NIH

Page 6: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Translational and

Clinical Sciences

Cardiovascular and

Respiratory Sciences

Surgical Sciences,

Biomedical

Imaging and

Bioengineering

Musculoskeletal, Oral Musculoskeletal, Oral

And Skin Sciences And Skin Sciences

Oncology 2 – Oncology 2 –

Translational Clinical Translational Clinical

Vascular and Vascular and

HematologyHematology

Physiological and

Pathological Sciences

Endocrinology, Endocrinology,

Metabolism, Metabolism,

Nutrition &Nutrition &

Reproductive SciencesReproductive Sciences

ImmunologyImmunology

Infectious DiseasesInfectious Diseases

& Microbiology& Microbiology

Digestive, Kidney &

Urological Systems

Neuroscience, Development

and Aging

Brain Disorders &Brain Disorders &

Clinical NeuroscienceClinical Neuroscience

Molecular, Cellular &Molecular, Cellular &

Developmental NeurosciencDevelopmental Neuroscience

Integrative, Functional & Integrative, Functional &

Cognitive NeuroscienceCognitive Neuroscience

Emerging Technologies &Emerging Technologies &

Training in NeuroscienceTraining in Neuroscience

Biology of Development

and Aging

Biobehavioral &Biobehavioral &

Behavioral ProcessesBehavioral Processes

Risk, Prevention& Risk, Prevention&

Health Health BehaviorsBehaviors

Epidemiology & Epidemiology &

Population Sciences Population Sciences

Healthcare Delivery Healthcare Delivery

& Methodologies& Methodologies

AIDS &AIDS &

Related ResearchRelated Research

AIDS, Behavioral

and Population SciencesBasic and Integrative

Biological Sciences

Biological Chemistry & Biological Chemistry &

Macromolecular Macromolecular

Biophysics Biophysics

Bioengineering SciencesBioengineering Sciences

& Technologies& Technologies

Genes, Genomes Genes, Genomes

& Genetics & Genetics

Oncology 1 – Basic Oncology 1 – Basic

TranslationalTranslational

Cell BiologyCell Biology

Interdisciplinary Interdisciplinary

Molecular Molecular

& Training& Training

CSR Review Divisions

Page 7: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Help Get Your Application to the Right Study SectionHelp Get Your Application to the Right Study Section

• Review CSR Integrated Review Group and Scientific Review Group (Study Section) guidelines to identify a home for your application.

• Submit a Cover Letter!

Page 8: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Help Get Your Application to the Right Study SectionHelp Get Your Application to the Right Study Section

http://www.csr.nih.gov/

Page 9: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Help Get Your Application to the Right Study SectionHelp Get Your Application to the Right Study Section

Integrated Review Group

Page 10: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Help Get Your Application to the Right Study SectionHelp Get Your Application to the Right Study Section

Study Section

Page 11: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Cardiovascular and Respiratory Sciences IRGCardiovascular and Respiratory Sciences IRG

• Cardiac Contractility, Hypertrophy and Failure• Cardiovascular Differentiation and

Development (CDD)• Clinical and Integrative Cardiovascular

Sciences (CICS) • Electrical Signaling, Ion Transport and

Arrhythmias (ESTA) • Myocardial Ischemia and Metabolism (MIM)• Lung Injury, Repair and Remodeling (LIRR)• Lung Cellular, Molecular and Immunobiology• Respiratory Integrative Biology and

Translational Research (RIBT)

Page 12: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Surgical, Biomedical Imaging and Bioengineering IRGSurgical, Biomedical Imaging and Bioengineering IRG

• Clinical, Molecular Imaging and Probe Development (CMIP)

• Medical Imaging (MEDI)

• Surgery, Anesthesiology and Trauma (SAT)

• Bioengineering, Technology and Surgical Sciences (BTSS)

• Biomedical Imaging Technology A & B (BMIT)

Page 13: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Division of Translational and Clinical Sciences Division of Translational and Clinical Sciences (DTCS) Trans-IRG Imaging SEP(DTCS) Trans-IRG Imaging SEP

• Goal - enhance review of multidisciplinary, translational research with a focus on imaging

Transformative in nature (translational research)

Include humans

Use imaging as a measure of diagnosis, intervention or treatment success/efficacy

• Editorial Board (Two Stage) ReviewFirst stage – 3 mail reviews (technical focus)

Second stage – 3 clinical impact reviews

• Crosses disciplines of cancer, cardiovascular, musculoskeletal and neuroscience

Page 14: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Submit a Cover LetterSubmit a Cover Letter

The cover letter should be used for a number of important purposes:

•Suggest Institute/Center assignment

•Suggest review assignment

•Identify individuals in conflict

•Identify areas of expertise needed to evaluate the application

•Discuss any special situations

It is NOT appropriate to use the cover letter to suggest specific reviewers.

Page 15: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Peer Review in CSRPeer Review in CSR

• CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section.

• Each CSR standing study section has 12-25 regular members who are from the scientific community.

• Temporary members are recruited as needed.

• About 60-100 applications are normally reviewed at each study section meeting.

Page 16: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

The Role of Your Scientific Review The Role of Your Scientific Review OfficerOfficer

Designated Federal Official responsible for the overall review process

Page 17: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Your Study Section Meeting Your Study Section Meeting

Page 18: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Your Application Is Reviewed In OrderYour Application Is Reviewed In Order

Clustered and Ordered for Fairness

•New and Early Stage Investigator applications are usually reviewed first

•Applications in each cluster are reviewed in order of their preliminary scores

Page 19: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

• New Investigator (NI): New Investigator (NI):

PD/PI who has not yet competed successfully for a substantial NIH research grant

o For multiple PD/PIs-all PD/PIs must meet requirements for NI status

• Early Stage Investigator (ESI):Early Stage Investigator (ESI):

PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

Page 20: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

• Reviewers discuss about 50-60% of the applications

• The panel will discuss any application a reviewer wants to discuss

Discussions Focus on the Best Applications Discussions Focus on the Best Applications

Page 21: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Additional Review PlatformsAdditional Review Platforms

• Helps to recruit reviewers

• Electronic review modes reduce travel

• Electronic ReviewsTelephone Assisted Meetings

Video Assisted Meetings

Telepresence

Internet Assisted Meetings

• Editorial Board Review – multidisciplinary

Page 22: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Review CriteriaReview Criteria

• Overall Impact: Assessment of the likelihood for the project to exert a sustained, powerful influence on the exert a sustained, powerful influence on the research field(s) involvedresearch field(s) involved

• Core CriteriaSignificance

Investigator(s)

Innovation

Approach

Environmento Review criteria each scored from 1-9Review criteria each scored from 1-9

Page 23: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

How Your Application Is Aligned to the ReviewHow Your Application Is Aligned to the Review

Criteria Application Section

Significance Research Strategy a. Significance

Investigator(s) Biosketch Personal Statement

Innovation Research Strategy b. Innovation

Approach Research Strategy c. Approach

Environment Resources Environment

Page 24: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

9-Point Scoring Scale

Impact Score Descriptor

High Impact

1 Exceptional

2 Outstanding

3 Excellent

Medium Impact

4 Very Good

5 Good

6 Satisfactory

Low Impact

7 Fair

8 Marginal

9 Poor

Page 25: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

• Scores for each review criterion

• Essentially unedited critiques

• Administrative notes if any

If your application is discussed, you also will receive:

• An overall impact/priority score and percentile ranking

• An Summary of review discussion

• Budget recommendations

Your Summary StatementYour Summary Statement

Page 26: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Summary Statement

New Indicator for Early Stage Investigators

Percentile: 29 Percentile in whole numbers

Impact/Priority Score 10-90 range

Program Officer

Page 27: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

• Read instructions

• Never assume that reviewers will know what you mean

• Refer to literature thoroughly

• State rationale of proposed investigation

• Include well-designed tables and figures

• Present an organized, lucid write-up

• Obtain pre-review from faculty at your institution

NIH Grant Writing Tips

http://grants.nih.gov/grants/grant_tips.htm

When Preparing an ApplicationWhen Preparing an Application

Page 28: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Additional Factors to ConsiderAdditional Factors to Consider

• Focus on significance/impact• Make it exciting• Be very clear• Do not assume too much• Have realistic aims and timelines -- Don’t be too

ambitious• Be brief with things that everybody knows• Note the study’s limitations• Proofread the application

Page 29: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Who Can Answer Your Questions?Who Can Answer Your Questions?

Before You Submit Your Application

• A Program Officer at an NIH Institute or Center

After You Submit

• Your Scientific Review Officer

After Your Review

• Your Assigned Program Officer

Page 30: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Key NIH Review and Grants Web SitesKey NIH Review and Grants Web Sites

NIH Center for Scientific Review

http://www.csr.nih.gov

NIH Office of Extramural Research

http://grants.nih.gov/

Page 31: An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association.

Helpful HandoutsHelpful Handouts

Insiders Guide What Happens to NIH Grant Application to Peer Review Your Grant Application Useful Web Links

http://cms.csr.nih.gov/publications/

http://cms.csr.nih.gov/publications/

Critical book review do’s and don’ts

Critical book review do’s and don’ts

Do’s and Don’ts of Employee Recordkeeping

Do’s and Don’ts of Employee Recordkeeping

Gekwetste jongeren: do’s and don’ts

Gekwetste jongeren: do’s and don’ts

Do’s and don’ts of presenting research

Do’s and don’ts of presenting research

Pupil Transportation Do’s and Don’ts

Pupil Transportation Do’s and Don’ts

The do’s and don’ts of resumes

The do’s and don’ts of resumes

DirectAccess, do’s and don’ts

DirectAccess, do’s and don’ts

Multisite WordPres Network Do’s & Don’ts

Multisite WordPres Network Do’s & Don’ts

Demolition Do’s & Don’ts

Demolition Do’s & Don’ts

PowerPoint's The do’s and don’ts

PowerPoint's The do’s and don’ts

PHARMACY AUTOMATION DO’S AND DON’TS

PHARMACY AUTOMATION DO’S AND DON’TS

Graph Modeling Do’s and Don’ts - Patrick Durusau Graph Modeling Do’s and Don’ts @markhneedham mark.needham@neotechnology.com

Graph Modeling Do’s and Don’ts - Patrick Durusau Graph Modeling Do’s and Don’ts @markhneedham [email protected]

Do’s and Don’ts in SEO

Do’s and Don’ts in SEO

User Adoption Do’s and Don’ts

User Adoption Do’s and Don’ts

Do’s & Don’ts

Do’s & Don’ts

Do’s and Don’ts when Visiting Italy

Do’s and Don’ts when Visiting Italy

Do’s & don’ts of interview

Do’s & don’ts of interview

Baby Naps Do’s and Don’ts

Baby Naps Do’s and Don’ts

Oral Presentation Do’s & Don’ts! Do’s & Don’ts!. First, … the “Do’s” Speak clearly Use large fonts. Use large fonts. Anything smaller than 24 point.

Oral Presentation Do’s & Don’ts! Do’s & Don’ts!. First, … the “Do’s” Speak clearly Use large fonts. Use large fonts. Anything smaller than 24 point.

WordPress Multisite Network Do’s & Don’ts

WordPress Multisite Network Do’s & Don’ts