“Advocates for Open Access Call for the Release of Evidence to Support Transparency”

We are proponents of utilizing evidence to enhance education. That’s why we authored a publication on the subject. However, when it pertains to advocating for more and superior evidence usage in the sector, increased mandates may not always be advantageous.

The U.S. Department of Education is revising the Education Department General Administrative Regulations (EDGAR), which dictate how the agency dispenses and oversees grants. While the majority of the proposed revisions are administrative, ED is seizing this opportunity to incorporate some significant alterations to evidence regulations. Among other changes, they intend to apply the evidence tiers specified in the Every Student Succeeds Act to all ED elementary and secondary programs, including competitive grants programs, where proposals supported by the preferred evidence types under the law may receive special consideration.

ESSA was the initial federal education legislation to define “evidence-based” and to differentiate between activities based on the strength of their research backing. The law mandates that school districts incorporate activities meeting at least the “promising” tier when using federal funds to intervene in struggling schools. It also mandates the department to prioritize evidence-supported proposals when disbursing funds through seven competitive grant programs. ED’s proposed regulations aim to extend this framework across the agency’s entire range, providing the Department with the option to prioritize such proposals in any of its competitive grant programs.

The challenge lies in the fact that, to adhere to the essence, not just the wording, of the evidence requirements, state and district personnel would necessitate more time and expertise than they presently possess. One of us, Carrie, held the position of research director in a state education agency for more than ten years and had the chance to witness educators’ proficiency in evidence utilization firsthand. Virtually every educator she collaborated with aspired to leverage evidence in shaping their practices but had limited knowledge on how to do so.

The competencies needed for effective evidence utilization are scarce, and the capabilities of districts and states vary significantly. Some agencies have heavily invested in research personnel and external partnerships. Others possess the ability to fulfill the requirements if regarded as a narrowly defined compliance task: filling out a logic model or identifying a study to support their proposed initiatives. Nonetheless, even this checkbox-style evidence utilization demands time and expertise, and numerous agencies struggle to even reach this stage. Hence, heightening evidence requirements would accentuate existing disproportions in agency resources by fostering unequal access to competitive federal funding.

Some may argue that this low and erratic capacity issue could be addressed through robust policy guidance. Since the enactment of ESSA in 2015, ED has endeavored to do so through its counsel on the statute’s evidence requirements. While the guidance is detailed and reflective, districts and states still encounter challenges in executing the mandates. How would broadening the requirements to encompass all federal grants lead to a more favorable result? No matter how meticulously crafted, policy guidance alone cannot actualize the proposed evidence regulations in practice. This is because policy guidance cannot furnish the genuine missing ingredients: time and training.

Even if all implementation hurdles were surmounted, practitioners would still confront an evidence base in education unsuited for the decisions they must make. These decisions delve deeper than the usual research inquiries: How should instructional time be distributed across standards in a particular grade and subject? To assist students lagging behind grade level, should resources be allocated to intensified math classes or concentrated holiday-week programs? In what manner should the teacher coaching framework be structured to yield optimal benefits for teachers?

While research may offer insights, studies are seldom designed to conclusively answer such nuanced questions. Consequently, the evidence repository on topics concerning practitioners is often lacking, particularly in the higher tiers of rigor, and the settings where the studies were conducted might limit the applicability of the findings to other contexts. When practitioners compose grant proposals for programs regulated by EDGAR, they could be nudged by the revamped evidence requirements to propose projects addressing less crucial issues but supported by robust research. Nonetheless, they might achieve better educational outcomes by suggesting a less-explored approach tackling a more urgent need or better suited to their local circumstances.

The proposed revisions to EDGAR are well-intentioned, but their consequences are foreseeable: increased paperwork, inequitable outcomes, and minimal progress in fostering genuine evidence utilization in educational environments. Fortunately, there is still an opportunity to influence the Department to alter its course: Public feedback on the proposal is due by February 26. We have expressed our concerns and urge others in the education research and practice communities to do the same.

If we desire practitioners to engage in evidence work effectively, the emphasis should be on support rather than compliance. Keep EDGAR out of the equation.

Carrie Conaway is senior lecturer at the Harvard Graduate School of Education. Nora Gordon is professor of public policy at Georgetown University. They are the co-authors of Common-Sense Evidence: The Education Leaders’ Guide to Using Data and Research (Harvard Education Press 2020).