IES Grant Writing Webinar

The Institute of Education Sciences (IES) Grant Writing webinar held on June 17 provided instructions and recommendations specifically for Education Research Grants Program (84.305A) and Special Education Research Grants Program (84.324A).

The webinar emphasized that applicants should email IES Program Officers early in the process to discuss the proposed framework and statement of purpose. IES program staff can review draft proposals and provide feedback as time permits. Additionally, the webinar discussed parts of the proposal, key problems to avoid, and writing tips among other topics.

Reminder: The final webinar in this series will be held Thursday, July 14 as follows:

IES Application Process
Thursday, July 14, 11:00 a.m. – 12:30 p.m. EDT
IES staff will provide information regarding the grant submission process. Topics focus on the application instructions, including content and formatting requirements, registration and submission through Grants.gov, and application forms.
To register for the webinar, please click here.

Agenda
Grant Writing Tips
General Requirements
Grant Research Topics
Grant Research Goals
Four Sections of the Project Narrative
Peer Review Process

Visit the IES website Previous IES Research Funding Opportunities Webinars page to download the complete PowerPoint presentations and transcripts for FY 2017 webinars.

For more information, visit the IES Resources for Researchers.

RFA Changes of Note

For 84.305A:

• The Development and Innovation goal is back.
• The Improving Education Systems: Policy, Organization, Management and Leadership topic has been split into two separate topics: (1) Improving Education Systems and (2) Education Leadership.
• Three Special Topics (Arts in Education, Career and Technical Education, and Systemic Approaches to Educating Highly Mobile Students) are being competed in FY 2017.
• You have the option of using SciENcv to create an IES Biosketch for each key person and significant contributor on the project.

For 84.324A:

• For FY 2017, there is a focus on teachers and other instructional personnel across all topics and goals.
  • No related services personnel, except in Early Intervention
  • For Exploration projects, research can involve the study of pre-service teachers across all topics
• Maximum amount of funding that can be requested under each research goal is slightly reduced.
• Language has been added for applicants proposing SMART designs under Efficacy & Replication and Effectiveness goals.
• You have the option of using SciENcv to create an IES Biosketch for each key person and significant contributor on the project.

Grant Writing Tips

Think of your proposal as a persuasive essay. You know the problem and have the best way to address it. The opening paragraph sets the scene for the readers:

• It identifies the significance of the work to be done and what actually will be done.
• Readers use it to organize information in the rest of the application.
• You can lose your readers right off with an unclear opening.

The statement of purpose should

• Be short and attention-getting
• Contain the problem statement and your contribution to solving it
• Your fellow researchers, friends, and family members should be able to understand it and see its relevance

Theory of Change (i.e., Logic Model or Logical Framework) should be

• The model underlying your research
• A roadmap to your project narrative
• A source for generating research questions
• Constantly evolving

Theory of Change and the Research Plan
In your research plan, you need to specify exactly what it is you’re exploring, creating, validating, or testing. You also need to specify how you will do these things.

• Strategies/Activities: What are the pieces that you’ll be exploring, creating, testing, etc.?

• Outcomes
  • Indicators: What will you measure, and how you will measure it?
  • Populations: Who and where (both in treatment and control/comparison)?
  • Thresholds: What effect (size) should you expect?
  • Timeline: When should you be collecting what data?

Clarity of Writing
Don’t assume reviewers know what you are talking about. Reviewers get frustrated when they can’t understand what you are saying.

• Readers (e.g., application reviewers) often complain about lack of clarity as follows:
  • Significance too general
  • Lack of detail regarding intervention, development cycle, or data analysis
  • Use of jargon and assumptions of knowledge
  • Poor writing (e.g., grammar), awkward constructions, etc.

General Requirements

All proposed studies must

• Measure student education outcomes
• Be relevant to education in the U.S.
• Address authentic education settings
• Specify one research topic
• Specify one research goal

NCER Ultimate Outcomes of Interest: Student Outcomes

Grade	Outcome
Prekindergarten	School readiness (e.g., pre-reading, language, vocabulary, early math and science knowledge, social and behavioral competencies)
Kindergarten – Grade 12	Learning, achievement, and higher-order thinking in reading, writing, mathematics, and science; progress through the education system (e.g., course and grade completion or retention, high school graduation, and dropout); social skills, attitudes, and behaviors that support learning in school

Grade	Outcome
Postsecondary (Grades 13 – 16)	Access to, persistence in, progress through, and completion of postsecondary education; for students in developmental programs, additional outcomes include achievement in reading, writing, English language proficiency, and mathematics
Adult Education (Adult Basic Education, Adult Secondary Education, Adult ESL, and GED preparation)	Student achievement in reading, writing, and mathematics; access to, persistence in, progress through, and completion of adult education programs

NCSER Student Outcomes of Interest
(For FY 2017, primary outcomes of teachers/instructional personnel)

Grade	Outcome
Birth – 5	Developmental outcomes and school readiness
Kindergarten – High School	Achievement in core academic content (reading, writing, mathematics, science); behaviors that support learning in academic contexts; functional outcomes that improve educational results; and transitions to employment, independent living, and postsecondary education

Grant Research Topics

 Grant Topics

• All applications to the primary research grant programs must be directed to a specific topic:
  • Note on SF 424 Form, Item 4b (Agency Identifier Number)
  • Note at top of Abstract and Project Narrative
• Must address student education outcomes
• Grade range varies by topic
• Your project might fit in more than one topic

305A: Topics and Their Grade Range

Topic	Pre-kindergarten	K-12	Sub-Baccalaureate & Baccalaureate
Early Learning Programs and Policies	X
Cognition & Student Learning	X	X
Education Technology	X	X
Education Leadership		X
Effective Teachers & Effective Teaching		X
English Learners		X
Improving Education Systems		X
Mathematics & Science Education		X
Reading & Writing		X
Social & Behavioral Context for Academic Learning		X
Special Topics		X
Postsecondary & Adult Education			X

324A: Topics and Their Grade Range

Topic	Infants, Toddlers, & Preschoolers	K-12
Early  Intervention & Early Learning in SpEd	X
Autism Spectrum Disorders		X
Cognition & Student Learning in SpEd		X
Families of Children with Disabilities		X
Mathematics & Science Education		X
PD for Teachers & Related Services Providers		X
Reading, Writing & Language Development		X
Social & Behavioral Outcomes to Support Learning		X
SpEd Policy, Finance, and Systems		X
Technology for Special Education		X
Transition Outcomes for Secondary Students with Disabilities		X

Choosing among Overlapping Topics

• What literature are you citing?
• To which topic is your area of expertise best aligned?
• If your focus is on a specific population of students/teachers, go to that program/topic:
  • Is your focus on a specific type of student/teacher (e.g., English Learners), or are you studying them as a subgroup of your sample?

Grant Research Goals

Research Goals

• All applications to 84.305A/84.324A must be directed to a specific goal:
  • Note on SF 424 Form, Item 4b
  • Note at top of Abstract and Research Narrative
• The goal describes the type of research to be done
• Every application is directed to a specific topic/goal combination

FY 2017 Research Goals

• Exploration
• Development & Innovation
• Efficacy & Replication
• Effectiveness
• Measurement

Maximum Award Amounts

Goal	Maximum Grant Duration	Maximum Grant Award
Exploration – With secondary data – With primary data	2 years 4 years	$600,000 $1,400,000
Development & Innovation	4 years	$1,400,000
Efficacy & Replication – Efficacy or Replication – Follow-up study – Retrospective	5 years (4 years NCSER) 3 years 3 years	$3,300,000 $1,100,000 $700,000
Effectiveness – Effectiveness – Follow-up study	5 years 3 years	$3,800,000 $1,400,000
Measurement	4 years	$1,400,000

Goal Requirements

• Your application must meet all Project Narrative and Award requirements listed for the goal you select in order for your application to be considered responsive and sent forward to review.
• We strongly encourage you to incorporate the recommendations into your Project Narrative.
• All applications must include a Dissemination Plan.

Exploration Projects: Key Features

• Malleable factors must be under the control of the education system
• Something that can be changed by the system
• Examples
  • Student characteristics: behavior, skills
  • Teacher characteristics: practices, credentials
  • School characteristics: size, climate, organization
  • Education interventions: practices, curricula, instructional approaches, programs, and policies

Development & Innovation Projects: Key Features

• Iterative development process
• Well specified theory of change
• Data collected on feasibility and usability in authentic education settings
• Fidelity must be measured
• Pilot data on student outcomes

Efficacy & Replication Projects: Key Features

• Testing a causal question
• Ask what might be needed to implement intervention under routine practice, even if you intend to test under ideal conditions
• Consider role of developer to avoid conflict of interest for developer-evaluators
• Do not require confirmatory mediator analyses (primary research question) but recommend exploratory ones

Effectiveness Projects: Key Features

• IES expects researchers to
  • Implement intervention under routine practice
  • Include evaluators independent of development/distribution
  • Describe strong efficacy evidence for intervention (from at least one previous efficacy study)
• Does not expect wide generalizability from a single study
  • Expects multiple Effectiveness projects to this end
  • Sample size is not a key distinction from Efficacy
• Does not require confirmatory mediator analyses but encourages exploratory ones
• Cost of implementation is limited to 25% of budget

Measurement Projects: Key Features

• Assessments may also be developed in other goals, but not as the primary focus
• Primary product of measurement grant is the design, refinement, and/or validation of an assessment
• Include an assessment framework
• Must link the assessment to student education outcomes

Expected Products

• Expected Products for each goal can help you identify the right goal for your project
• At the end of a funded project, IES expects you to provide… (see RFA)

Four Sections of the Project Narrative

Project Narrative

• Four Required Sections
  • Significance
  • Research Plan
  • Personnel
  • Resources
• Each of these sections will be scored individually by the peer reviewers
• In addition, reviewers provide an overall score of Scientific Merit
• Requirements vary by Topic & Goal
• READ THE REQUIREMENTS CAREFULLY
• 25 pages, single spaced
• Project Narrative is supported by Appendices, but all critical content for reviewers should be included within the 25 pages of the Project Narrative

Significance Section

• Describes the overall project
  • Your research question to be answered, intervention to be developed or evaluated, or measure to be developed and/or validated
• Provides a compelling rationale for the project
  • Theoretical justification
    • Theory of Change
  • Empirical justification
  • Practical justification
• Do not assume reviewers know significance of your work
• Do not quote back RFA on general importance of a topic
  • e.g., RFA paragraph on lack of reading proficiency of 8^th and 12^th graders based on NAEP data
• Do quote back RFA if your project is addressing a research gap or consideration identified in the RFA
  • e.g., disproportionality in discipline (Social/Behavioral); impact of early childhood policy initiatives (Early Learning)

Significance: 2 Key Problem Areas

1. Description of Malleable Factor/Intervention
   • Unclear what intervention is: confuses reviewers
     • Many components and may be applied at different times – how fit together – Graphic may help
   • Unclear how to be implemented to ensure fidelity
   • Intervention not shown to be strong enough to expect an impact
     • Especially true for information interventions – e.g., provide data on students, short teacher workshops
   • Overly focused on actions not content
     • : 20 hours of PD held over 10 weeks but no detail on what is to be covered in the sessions

2. Theory of change
   • Why a malleable factor is expected to be related to a student outcome
   • Why the proposed intervention should improve outcomes versus current practice
   • How an assessment/instrument will measure a specific construct
   • When well laid out, a theory of change makes clear what is expected to happen and in what order
   • Easy for reviewers to understand research plan – why measure certain outcomes
   • Graphic can be helpful

Theory of Change Should Describe

• How the intervention addresses the need and why it should work
  • Content: what the student should know or be able to do; why this meets the need
  • Pedagogy: instructional techniques and methods to be used; why appropriate
  • Delivery System: how the intervention will arrange to deliver the instruction
  • Which aspects of the intervention are different from the counterfactual condition
• Key factors or core ingredients most essential and distinctive to the intervention
• Do not overwhelm the reader
• Do not use color as a key because applications are often reviewed in black and white

Research Plan Section

• Describe the work you intend to do
  • How you will answer your research question; develop your intervention; evaluate the intervention, or develop and/or validate your assessment
• Make certain Research Plan is aligned to Significance section
  • All research questions should have justification in Significance
• Step-by-step process
  • A timeline is strongly recommended!

Identify Setting, Population, & Sample

• Identify the places you will be doing research
• Identify the population you are addressing
• Identify the sample
  • Inclusion and exclusion criteria
  • Sample size (issues of power for analysis)
  • The importance of attrition and how to address it
  • External validity: can you generalize to your population or only to a subset of it
• If using secondary data, discuss these for the datasets you will be using

Specify Your Outcome Measures

• For both proximal and distal outcomes
• Sensitive (often narrow, aligned with intervention) measures
• Measures of broad interest to educators
• Describe reliability, validity, and relevance
• Do not include measures not linked to research questions
• Consider issue of multiple comparisons

Specify Features of All Other Measures

• Measures that feed back into iterative development process
• Fidelity of Implementation
  • Operating as intended
  • Able to address comparison groups
• Feasibility

Qualitative Measures

• Describe
  • Actual items to be used
  • How items link to constructs – the validity of these measures
  • Procedures for collection and coding (address inter-rater reliability)
  • How qualitatively collected measures are used in analysis of quantitative outcomes (e.g., test scores)

Measurement Projects

• Alternate forms – horizontal equating
• Vertical equating, if measuring growth
• Test fairness
• Non-student instruments must be validated against student outcomes

Analysis Depends on Design

• Describe how your analysis answers your research questions
• Describe analyses of qualitative data
• Show your model
• Identify coefficients of interest and their meaning
• Show different models for different analyses
• Include Equations
• Address clustering
• Describe plan for missing data
• Check for equivalency at start and attrition bias
• Use sensitivity tests of assumptions

Personnel Section

Describe key personnel:

• Show that every aspect of project has an individual with expertise to do it
  • Appropriate methodological expertise
  • Substantive person for all issues addressed
  • Do not propose to hire a key person with X expertise
  • Project management skills
• Show that every aspect of project has enough time from an expert

• Orient CVs so it is specific to the project
  • Can use SciENcv to create an IES Biosketch
  • 4 pages plus 1 page for other sources of support

Personnel Strategies for PI

• Senior Researcher as PI
  • Show adequate time to be PI
  • Make credentials clear (not all reviewers may know)
• Junior Researcher as PI
  • Show you have adequate expertise not only to do work but to manage project
    • Continuation of graduate/postdoctoral research
    • Management skills as graduate student/postdoc
  • Reviewers more comfortable if you have senior person(s) on project to turn to for advice
    • Co-PI, Co-I, contractors, advisory board
    • Have them on for enough time to be taken seriously

Resources

• Show institutions involved have capacity to support work
  • Do not use university boilerplate
• Show that all organizations involved understand and agree to their roles
  • What will each institution, including schools, contribute to the project
  • Show strong commitment of schools and districts and alternatives in case of attrition
• If you have received a prior grant award for similar work, describe the success of that work

Dissemination Plan

• Describe your capacity to disseminate information about the findings from your research
• Identify the audiences that you expect will benefit from your research
• Discuss the ways in which you intend to reach these audiences
• Appendix D should back up the Resources section
• Detailed Letters of Agreement from research institutions, States, districts, schools
• Do letters show that partners understand their role in the project (e.g., random assignment to condition, time commitments)?
• Do letters show that you have access to all necessary data to do the proposed work?

Budget and Budget Narrative

• Provide a clear budget and budget narrative for overall project and each sub-award
• Provide detail on the assumptions used in the budget (e.g., assumptions for travel)
• Budget categories are described beginning on pg. 110 of 84.305A RFA, and pg. 116 of 84.324A RFA
• Check RFA for specific budget requirements for Research Goals
• Ensure alignment among Project Narrative, Budget, and Budget Narrative
• Level of effort should be in 12-month calendar year

Appendices

Appendix	Content to be included
A	Response to reviewers for resubmitted applications – Limit 3 pages
B	Figures, charts, tables that supplement project narrative; examples of measures to be used in the project (optional) – Limit 15 pages
C	Examples of materials to be used in the intervention or assessment that is the focus of your project (optional) – Limit 10 pages
D	Letters of agreement from partners, data sources, and consultants (optional) – No page limit
E	Data Management Plan (Required for Efficacy/Replication and Effectiveness Applications) – Limit 5 pages

Peer Review Process

Peer Review:

• Applications are reviewed for compliance and responsiveness to the RFA
• Applications that are compliant and responsive are assigned to a review panel
• Two or three panel members conduct a primary review of each application
• The most competitive applications are reviewed and discussed by the full panel
• Note: Scientific Review Office is separate from Program Officers in NCER and NCSER
• Peer Review of Grants Process
  • http://ies.ed.gov/director/sro/peer_review/application_review.asp
• Lists of Peer Reviewers
  • http://ies.ed.gov/director/sro/peer_review/reviewers.asp