Fifteen years ago, when Neil Heffernan began developing the online math homework system ASSISTments, he knew he faced an uphill battle in presenting his program to skeptical school districts. Administrators, teachers, and even parents are wary of pricey computer-based “ed-tech” programs that are not proven to produce reliably measurable results. He knew that even though ASSISTments is offered free, he would need proof that his program was different and that it worked.
“Schools have to be cautious in adopting new programs for their students, and when you say something is free, they look at you like you’re a snake oil salesman,” says Heffernan, professor of computer science and director of the Learning Sciences and Technologies PhD program. “We had seen firsthand the effect of ASSISTments on teachers and students in every school we worked with, but we didn’t yet have hard data to show it.”
Now, he does.
U.S. Education Secretary John King Jr., left,
and Neil Heffernan observe students using
ASSISTments at an event at the White
House in late 2016.
Results of a randomized control trial conducted in public schools in Maine and published today in the journal AERA Open by the American Education Research Association clearly quantify the program’s benefits: “Students in schools using ASSISTments scored approximately 75 percent higher than students in schools that were not using the program,” says Heffernan.
The study conducted by SRI Educational Research stemmed from a $3.5 million grant from the Institute of Education Sciences (IES) of the U.S. Department of Education, which awarded to SRI in 2012 to evaluate the effectiveness of ASSISTments for mathematics homework. The University of Maine assisted with the study by conducting classroom observations and other research.
The study examined test scores of 2,850 seventh graders at 43 public schools in the state of Maine, which has provided laptop computers or iPads to 7th and 8th graders in all schools across the state since 2002 under a program begun by then-governor Angus King. Schools were matched in terms of demographics and socio-economic status and then randomly assigned by SRI to the test group or control group.
According to national data provided by test publisher McGraw Hill, TerraNova scores generally increase by an average of 11.66 points per year between 6th and 9th grades. In the Maine study, however, students using ASSISTments scored an average of 8.84 points higher than those in the control group. Presuming the control group in Maine met the national average, the added 8.84 points gained by students using ASSISTments is a large and meaningful effect.
“Smaller, earlier studies had shown solid gains for students who use the system, but schools naturally wanted hard, objective evidence that their students’ scores would improve,” Heffernan says. “This rigorous, randomized study by SRI—a very well respected educational research organization—has given us the gold-standard proof schools want.”
ASSISTments allows students to receive immediate feedback as they complete their homework.
Notably, ASSISTments proved most beneficial for students who began the year with lower math scores—which is perhaps the most rewarding finding for Heffernan.
“Of the schools in Maine using ASSISTments, some 60 percent of them are in towns that are of lower socio-economic status where students tended to have lower scores,” Heffernan notes. “Using ASSISTments essentially brought those students up to the level of their peers in towns of higher socio-economic status.”
In addition, teachers reported they found the system easy to implement, in part because it fits within common school policies for homework. Using ASSISTments doesn’t require them to change their teaching or their pacing through topics, or adopt new textbooks. Teachers assign homework in the platform to match their needs.
“Technology is the amplifier” of teacher interactions with students, “and the SRI study shows it can be a very powerful amplifier,” says Heffernan, who got the idea for ASSISTments after watching his then-fiancée Cristina Lindquist tutor a student. In response to the student’s attempts to solve problems, Lindquist (now Cristina Heffernan) provided coaching and hints that helped the student better understand his missteps, prompting him to rethink his approach and make better progress. Heffernan videotaped the interactions and analyzed the series of steps both tutor and student took.
A key differentiator, ASSISTments begins with the teacher selecting the assignment. It then generates meaningful dashboard reports that quickly show teachers where a student—or many students—may be having a problem This allows for a more efficient review of problem areas and allows the teacher to help students progress more smoothly through the material. (The system’s name is a blend of “assistance” and “assessment;” the program assists students and teachers by assessing student progress on each educational topic.)
ASSISTments has been used by 100,000 elementary, middle, and high school students across the U.S. since 2002. Heffernan and colleagues have received more than $16 million in funding over the past decade to develop, implement, research, and update the system. They are currently working to expand ASSISTments to include crowdsourcing as a tool to involve more teachers and researchers in the development process. In addition, WPI will provide ASSISTments with funding to support two more programmers to help expand its reach.
Aside from nominal training and set-up fees to cover the team’s expenses, ASSISTments is available as a free public service of WPI. For more information, go to www.assistments.org.
- By Alison Duffy
