Securing External Federal Funding
description
Transcript of Securing External Federal Funding
![Page 1: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/1.jpg)
Securing External Federal FundingJanice F. Almasi, Ph.D.
Carol Lee Robertson Endowed Professor of Literacy
University of Kentucky
![Page 2: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/2.jpg)
Institute of Education Sciences2 Grant Competitions per year
Janice F. Almasi, Ph.D., University of Kentucky
![Page 3: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/3.jpg)
Current Funding Opportunities14 Long-term Programs of Research
Drag picture to placeholder or click icon to add
![Page 4: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/4.jpg)
![Page 5: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/5.jpg)
Be InformedSubscribe to Newsflash at ies.ed.gov/newsflash
![Page 6: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/6.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
IES Research Goals
Goal 1: Identification Identifying programs and practices associated with
better educational outcomes (secondary data analysis) Goal 2: Development Projects
Developing educational interventions Goal 3: Efficacy and Replication Projects
Determine if fully-developed interventions are effective Goal 4: Scale-Up Goal 5: Measurement Projects
![Page 7: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/7.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Prior to Peer Review Meeting
Triage identifies top 25 applications
Reviewers read, rate about 8 applications
Reviewers check for COIs
Each application assigned to at least 2 reviewers
![Page 8: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/8.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Criteria
Significance Research Plan Personnel Resources
![Page 9: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/9.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Review Criterion Ratings(M
ore
Weak
ness
es
than
St
reng
ths)
(Bala
nce
of
Stre
ngth
s and
We
akne
sses
)
(Mor
e St
reng
ths t
han
Weak
ness
es)
1
Poor Excellent
765432
![Page 10: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/10.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
What Reviewers Look ForWhere Applications Tend to be Weak
![Page 11: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/11.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Significance
Goal 1• Theoretical and
empirical rationale for study and practical importance of the intervention (e.g., program, practice) that will be examined
Goals 2 and 3• Describe (a) the
intervention (e.g., features, components) and the logic model for the intervention, (b) theoretical and empirical support for intervention, and (c) practical importance of the intervention
![Page 12: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/12.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Significance in Goals 2 and 3
Context for Proposed Interventions Provide context for the proposed intervention by including
data on, or reviewing research describing, the attributes of typical existing practices.
Identify shortcomings of current practice and how they contribute to the rationale for the proposed intervention.
Provide context for understanding how much of a change the proposed intervention is intended to achieve.
![Page 13: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/13.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Significance in Goals 2 and 3 Intervention, Theory of Change, Empirical/Theoretical Rationale
Clearly describe the intervention Clearly describe the theory of change for the intervention
How do the features or components of the intervention relate to each other temporally (or operationally), pedagogically, and theoretically (e.g., why A leads to B)?
Provide a strong theoretical and empirical justification for the design and sequencing of the features or components of the intervention. Enables evaluation of: Relation between the intervention and its theoretical and
empirical foundation (e.g., is the proposed intervention a reasonable operationalization of the theory?)
Relation between the intervention and the outcome measures (e.g., do the proposed measures tap the constructs that the intervention is intended to address?)
Include a Logic Model
![Page 14: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/14.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Significance in Goals 2 and 3
Practical Importance of Intervention When the proposed intervention is fully developed will it
have the potential to improve student outcomes in educationally meaningful increments, if it were implemented over the course of a semester or school year?
Would the proposed intervention be both affordable for and easily implemented by schools (e.g., not involve major adjustments to normal school schedules)?
![Page 15: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/15.jpg)
Janice F. Almasi, Ph.D., University of KentuckyResearch Plan: Goal 2
Sample• Samples and settings used to
assess feasibility of intervention and for pilot data assessing promise of intervention
Iterative Development Process• Revision• Implementation• Observation• Revision
How do you define“operating as intended?”
What data will begathered to determine
how intervention isoperating?
How will the datagathered be used
to revise the intervention?
What criteria will be used to
determine if the intervention operates
as intended?
![Page 16: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/16.jpg)
Janice F. Almasi, Ph.D., University of KentuckyResearch Plan:Goal 2
Feasibility of Implementation• Goal is a fully developed intervention• Data that addresses feasibility of
implementing in small sample of authentic education settings
• Promise of intervention in terms of outcomesPilot Study
• Pilot data on outcome measures progressing in right direction
• Pilot data demonstrates implementation of intervention is associated with behaviors consistent with theory of change
• No more than 30% of funds• Data should not be a test of efficacy
![Page 17: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/17.jpg)
Janice F. Almasi, Ph.D., University of KentuckyResearch Plan:Goal 2
Measures• Clearly describe procedures for
gathering data to refine and revise the intervention and provide insight into feasibility and usability of proposed intervention• What needs to be observed?• How will observations be
gathered?• Clearly describe measures that will
be used (and reliability and validity if appropriate)
![Page 18: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/18.jpg)
Janice F. Almasi, Ph.D., University of KentuckyReearch Plan:Goal 3
Research Questions• Pose clear, concise hypotheses
or research questionsSample• Define sample to be selected• Define sampling procedures
(including justification for inclusion and exclusion)
• Strategies to be used to reduce attrition
![Page 19: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/19.jpg)
Janice F. Almasi, Ph.D., University of KentuckyReearch Plan:Goal 3
Research Design• Provide detail!• How will threats to internal/external validity be
addressed?• Studies using random assignment are preferred
where feasible• What is unit of randomization and what
procedures will be used to make assignments to conditions?
Power• What power is needed to detect a reasonably
expected and minimally important effect?• How was effect size calculated?• If clusters are randomly assigned to treatment
conditions be sure to include intraclass correlation and anticipated effect size in power analysis
![Page 20: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/20.jpg)
Janice F. Almasi, Ph.D., University of KentuckyResearch Plan:Goal 3
Measures• Justify appropriateness of measures• Are measures of practical interest to
educators and not overly aligned with intervention?
• Include reliability and validity informationFidelity of Implementation• How will implementation be documented and
measured?• How will factors associated with fidelity be
identified and assessed?• How will fidelity data be incorporated into
analyses of impact?• How do conditions in the school setting affect
fidelity of implentation?
![Page 21: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/21.jpg)
Janice F. Almasi, Ph.D., University of KentuckyResearch Plan:Goal 3
Comparison Group• How does comparison group compare to intervention
on critical features of intervention?• Using a “business-as-usual” comparison is
acceptable but explain why using it is acceptable• How will contamination be avoided?Mediating and Moderating Variables• Observational, survey, or qualitative methods are
encouraged to help identify factors that may explain the effect or lack of effect of intervention
Data Analysis• Quantitative: Specify statistical procedures and
include formulas where appropriate• Qualitative: Specific methods used to index,
summarize, and interpret data should be identified• Relation between hypotheses, measures, and
independent and dependent variables should be clear
![Page 22: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/22.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Personnel
What role will each individual have in the project? What qualifications, training, and experience do key
personnel possess? How will qualifications be used on the research? Are key personnel dedicating sufficient time to
competently implement proposed research?
![Page 23: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/23.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Resources
Are the resources adequate to support the proposed activities in terms of: Facilities Equipment Supplies Institutional Support for Managing/Directing Grants and
Supporting Scholarship
Have partners shown support for implementation and support of the project?
![Page 24: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/24.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Scientist Reviewer Critiques
A brief description of the overall application. Identify each application’s key strengths and
weaknesses in each of the evaluation areas and prepare critical, evaluative comments.
Integrated summary of the overall assessment of the application, including the main strengths and weaknesses of the application.
![Page 25: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/25.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Overall Score
Overall Score Range Adjectival Equivalent1.0 - 1.5 Outstanding1.6 - 2.0 Excellent2.1 - 2.5 Very Good2.6 - 3.0 Good3.1 - 4.0 Fair4.1 - 5.0 Poor
![Page 26: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/26.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Peer Review Meeting Process
3. Notetaker
Summarizes the discussion orally and in writing
2. Full PanelDiscusses the application, asks questions, and offers additional
critiqueDiscusses the budget
1. Assigned Scientist Reviewers Share Overall ScoresReviewer 1 Summarizes the
application and its strengths and weaknesses in each
Reviewer 2 elaborates on areas of agreement or disagreement
![Page 27: Securing External Federal Funding](https://reader036.fdocuments.net/reader036/viewer/2022062323/56815e72550346895dccf554/html5/thumbnails/27.jpg)
Janice F. Almasi, Ph.D., University of Kentucky
Peer Review Meeting Process
6. Assigned Scientist ReviewersMay edit/revise their original written critiques
based on panel discussion
5. Full PanelPrivately assigns criteria scores and overall scores
4. Assigned Scientist ReviewersAdjust initial recommended criteria scores and overall scores