Better Methods for Learning What Works.

Pointing the Way

Which reading program should we adopt? Which math program will raise student achievement? Rigorous research can point the way. Unfortunately, strong research evidence is not always available because randomized controlled trials (RCTs), which produce the strongest evidence, can be costly, difficult to conduct in school settings, and limited in their results.

Charting a Better Course

Researchers at SEDL and the University of Wisconsin-Madison are using innovative methods to overcome these hurdles. The team is conducting national RCTs of two widely used math and reading programs: McGraw-Hill Education’s Everyday Mathematics and Imagine It!, the newest edition of Open Court Reading.

The trials are evaluating the programs’ effectiveness at scale (i.e., as typically implemented) and across diverse populations and conditions in a large sample of elementary schools. Co-principal investigators are Michael Vaden-Kiernan, SEDL director of research and evaluation, and Geoffrey Borman, professor of education at the University of Wisconsin-Madison.

Instead of recruiting separate samples for each RCT, the team conceived an ingenious design to make the best use of funds and other resources. They are running the RCTs simultaneously so they can use one combined sample. The schools using the math program are serving as the control group for those using the reading program, and vice versa.

This design has enabled the team to increase the sample size, improving the RCTs’ ability to detect program effects under real conditions. As an added benefit, the combined sample ensures that every school receives a program, giving schools more incentive to participate.

To broaden the utility of the RCTs’ results, the team is also conducting implementation studies to examine how each program is implemented across various populations. The resulting information will enable the team to better interpret and explain the RCTs’ findings. For instance, the studies will shed light on how factors such as teacher experience may have affected outcomes.