Changes Are Coming to What Works Clearinghouse

Excerpted from Message from the IES Director

The Institute of Education Sciences (IES) is exploring ways to improve the What Works Clearinghouse (WWC), a flagship product of IES since its inception.

The WWC recently passed its 15th birthday and has gone through many changes since then to ensure that it continues to help the nation identify what works for whom and under what conditions.

Here are some of the general directions IES will be exploring:

• Increasing attention to the cost of interventions
  • Identifying what works is central to the mission of the WWC. But how much interventions cost is also critical. Two interventions with the same outcomes but different costs should be viewed differently by education administrators who must make spending decisions within budget constraints. IES has been paying more attention to cost in the last few years, and will continue to sharpen that focus in both the WWC and other IES work, including IES grants.

• Communicating more effectively to the range of stakeholders
  • IES’s authorizing legislation, the Education Sciences Reform Act (ESRA) of 2002, specifically charges IES with disseminating its work “in forms that are understandable, easily accessible, and usable, or adaptable for use in the improvement of educational practice by teachers, administrators, librarians, other practitioners, researchers, parents, policymakers, and the public…” IES has been focusing more on dissemination in the last few years, and we sharpen that focus in the WWC as well as in other IES work. IES will be revisiting the text and visual presentation of information throughout the WWC, including its intervention reports and practice guides, to make the information as accessible and useful as possible to all stakeholders.
  • IES will be systematically looking into making research metrics easier to understand. Here is an example of a concrete change that IES is pursuing: How can we best translate effect sizes into terms that policymakers and practitioners can readily understand, such as months gained on reading and math assessments or changes in the number of student absences or suspensions?
  • IES will also be looking at other metrics to see how they can be made more understandable. For example, as IES increases its attention to costs, it will be experimenting with new ways to present these data in an accessible and usable form.

• Reflecting updated practices with regard to statistical significance
  • The use of p=<.05 as a cutoff for statistical significance is a convention of long standing. Yet modern thinking and many new statistical approaches have outdated that bright line test. IES will be working on how to reflect these changes in how it judges research in the WWC. As IES does that, it will be ending the “substantively important” designation.
  • IES will also continue to advance our Single-Case Design standards and procedures. WWC now has “pilot standards” for Single-Case Design, a research approach central to much of the work supported by the National Center for Special Education Research (NCSER). IES will be working to see if the pilot status for Single-Case Design can be resolved.

• Broadening our coverage from preK–12
  • IES’s authorizing legislation is clear about our mandate to “provide national leadership in expanding fundamental knowledge and understanding of education from early childhood through postsecondary study…” The predominance of preK–12 research on the WWC does not comport with this mandate.
  • The WWC got into the postsecondary “business” relatively recently, and the paucity of research coverage fails to reflect the importance of that level of education and the centrality of postsecondary education to current policy. Similarly, there is far too little attention to what education pathways help students launch careers with good wages. IES will explore means of expanding WWC coverage in postsecondary education, especially with regard to career and technical training.
  • At the other end of the education spectrum, while funding for early childhood services is largely managed by the Department of Health and Human Services (HHS), IES’s authorizing legislation charges us with research on the educational development of children from birth to kindergarten entry. New and exciting brain research further underscores the importance of children’s learning starting at birth. Coordinating closely with the work of other agencies, such as the National Institutes of Health (NIH) and the Administration for Children and Families, we will seek to expand our current work to include this critical birth-to-three period as foundational to all children’s educational success from pre-K on.

IES will be consulting with a variety of stakeholders as it explores how best to refine and implement these and other changes.