Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer...
Transcript of Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer...
![Page 1: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/1.jpg)
Community Conversations Research Grant Review
Live Orientation LaunchApril 26th 2018
![Page 2: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/2.jpg)
Agenda
• Preparing for the Review
• Panel Information
• Review Participant Roles
• Review Form
• Sample Timeline
• Expectations for Comments and Clarification
• Next Steps
• Q&A
![Page 3: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/3.jpg)
Preparing for the Review
• All Reviewer resources are posted at:
– https://cnsgov.sharepoint.com/sites/extranet/OGPO/2018communityconversations/SitePages/Home.aspx
– Reviewers will have four mandatory trainings (calls/webinars)
• Launching the Review (current training)
• Reviewing the Applications-Part I
• Reviewing the Applications-Part II
• Ensuring an Equitable Review (External Reviewers only)
• Conflicts of Interest LMS training (Staff Reviewers only)
– Read competition documents:
• Notice of Funding
– Application Instructions
– FAQs
![Page 4: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/4.jpg)
Preparing for the Review
• Reading Materials
• Sample Application
• Review Materials:
– Review Form & Rubric
• Other reviewer resources and administrative forms on Reviewer Resource Webpage:
– Writing Meaningful Comments document
– Conflict of Interest and Participation Agreement Forms for External Reviewers
– Conflict of Interest Guidance and Participation Guidance for CNCS Staff Reviewers
![Page 5: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/5.jpg)
Panel Introduction emails will be sent Monday, April 30th
Approximately 10 Applications per panel
1 External Reviewer
1 CNCS Staff Reviewer
1 Program Office Liaison
1 GARP Liaison
The GARP Liaisons and Program Office Liaisons will each have multiple panels.
![Page 6: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/6.jpg)
Review Roles
• External Reviewer/Staff Reviewer
– Responsible for reading and assessing applications, engaging in panel discussion, addressing feedback, to complete high-quality review forms
• Program Office Liaison (POL)
– Serve as a resource to the panel on programmatic elements and answers content-specific questions
• GARP Liaison
– Manages review logistics for panels, ensures reviewers have completed requirements for orientation and access to materials, verifies final reviewer documents have been completed
![Page 7: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/7.jpg)
Review Roles
• Review Coordinators
– Program Office Lead: Andrea Robles, R&E
• POLs will be contacted as needed if specific selection criteria questions arise, programmatic questions about ratings or aspects on a particular criteria
– Office of Grants Policy and Operations Lead: Nikki Hayden and Susan Cohn, OGPO
• Manages review process
• Confirm successful review form submission
![Page 8: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/8.jpg)
Reviewer Responsibilities
• Review and score applications against selection
criteria and other considerations in the Notice
• Produce high quality assessments of the
strengths and weaknesses of the applications
• Help inform decision makers on quality of
applications to aid in their funding decisions
![Page 9: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/9.jpg)
IRF Development Process
Use Review Form to guide you through the criteria for assessing the application
AFTER initial IRF is completed, then meet with panel partner for Panel Discussion
Return to IRF to revise as necessary. Save IRF using Naming Convention, then send to GARP Liaison
Receive GARP Liaison feedback, revise & resubmit (if necessary) until approval is received
OGPO informs the Reviewer when Approval is granted on each application
![Page 10: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/10.jpg)
Panel Discussions
Consensus is not required of the panel discussion.
Reviewers are asked to engage in discussion
about the Criteria and consider the assessments
and finding of their fellow panel members. The
discussion should cover each of the relevant
elements of the application.
![Page 11: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/11.jpg)
Panel Discussion Report
![Page 12: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/12.jpg)
Panel Discussion Naming Convention
All review forms should follow the naming convention:
Legal Applicant Name. Application ID. PDR. Reviewer First Initial Last Name. Panel
Number.
Example:
UniversityofToronto.RX123456.PDR.ASmith.1
![Page 13: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/13.jpg)
Review Form Naming Conventions
All review forms should follow the naming convention:
Legal Applicant Name. Application ID. IRF. Reviewer First Initial Last Name. Panel
Number.
Example:
UniversityofToronto.RX123456.IRF.ASmith.1
![Page 14: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/14.jpg)
Submission of Review Materials
Both Panel Discussion Reports as well as Review Forms should be submitted to:
![Page 15: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/15.jpg)
Required Timeline Dates
Review Begins:
Monday, April 30th
Review Ends:
Monday, May 21st
All IRFs must be approved as final by OGPO by 5:00pm EST
![Page 16: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/16.jpg)
Week 1 Sample Timeline
• Monday, April 30th:
– All applications are received
– Reviewers have completed all orientations
– Begin reading applications for COI and follow up with GARP Liaison as needed
– Reach out via email to fellow panel member to schedule Panel Intro Call
![Page 17: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/17.jpg)
Week 1 Cont. Sample Timeline
• Tuesday, May 1st:
–Hold Panel Intro Call to establish order of reading applications and introduce panel members (this may also be covered via email exchange)
–Begin reading and completing IRF for three applications
• By Fri, May 4th
–First Panel Discussion: discuss 1-3 applications
–Send first IRF’s to GARP Liaison
![Page 18: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/18.jpg)
Week 2 Sample Timeline
• Wed, May 9th
– Receive feedback from GARP Liaison on first IRF
– Work with liaison to ensure first set IRF is approved
– Continue reading and reviewing applications 4-6
• Thu, May 10th:
– Second Panel Discussion: Discuss applications 4-6
• Fri, May 11th:
– Send IRFs for applications 4-6 to for GL review, and revise as necessary
– Continue reading and reviewing applications 7-10
![Page 19: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/19.jpg)
Week 3 Sample Timeline
• Tue, May 15th:
– Receive feedback from GL on applications 4-6
– Final Panel Discussion for applications 7-10
• Wed, May 16th:
– Send remaining IRFs to GL for review
• Fri, May 18th
– Receive feedback from GL on 7-10
• Mon, May 21:
– All IRFs must be accepted as final by 5:00 pm EST
![Page 20: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/20.jpg)
Tips for Producing High-Quality Review Forms
• Address only the Selection Criteria. Do not compare with other applications
• Address only the information within the application
• Use evaluative, grammatically correct language
• Comments and ratings should be aligned
![Page 21: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/21.jpg)
Writing Strong Comments
• Comments should align with/support Ratings
– Fair or Does Not Meet should result in a Weakness comment
– Good or Excellent should result in a Strength comment
• Comments should relate directly to the selection criteria
• Do not just reiterate the criteria or repeat summary from the application, explain how the applicant’s response to the criteria is strong or weak
• Use complete sentences
• Significant Strengths and Weaknesses are used as feedback to applicants
![Page 22: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/22.jpg)
Writing Items for Clarification
• Allows applicants to explain aspects of the application that were not clear, not intended as an option to provide additional information that should have been included in the initial application
• Allows applicants to adjust budget or other items that must be resolved prior to award
• Reviewers should always write Clarification as a clearly articulated question or statement requesting a response
![Page 23: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/23.jpg)
Next Steps: • Complete all Orientation Sessions/Readings
by Mon, April 30th –Verify completion at [email protected]
• Receive Panel Assignment email – (Monday, April 30th)
• Panel Intro Call/email – (Tue, May 1st )
• Review Check-In Q&A Calls – Fri, May 4th 1:00p.m. Eastern
– Fri, May 11th 1:00 p.m. Eastern
![Page 24: Community Conversations Research Grant Review the Review...•External Reviewer/Staff Reviewer –Responsible for reading and assessing applications, engaging in panel discussion,](https://reader030.fdocuments.net/reader030/viewer/2022040818/5e6361ca36842855b71c56dd/html5/thumbnails/24.jpg)
Questions?