Modifying homework procedures improved outcomes without altering curriculum
A new study offers evidence that simple and inexpensive changes to existing courses can help students learn more effectively.
The study from Rice University and Duke University found that making a few changes to homework assignments in an upper-level undergraduate engineering course at Rice led to improved scores on exams. The study appears this week in the journal Educational Psychology Review.
The findings by a team from Rice’s Center for Digital Learning and Scholarship and Duke’s Department of Psychology and Neuroscience demonstrate how technology and cognitive science can be combined to develop inexpensive but effective educational changes that required no changes to course curriculum.
“Based on laboratory studies, we know a lot about how people learn,” said lead author Andrew Butler, a postdoctoral researcher at Duke. “To test how well some of those cognitive science principles worked in a classroom, we made subtle changes to standard homework practice.”
“The results exceeded everyone’s expectations,” said Rice co-author Richard Baraniuk, the instructor of the upper-level “signals and systems” engineering course where the experiment took place. “These simple changes produced a larger effect than the average improvement for classroom interventions that require a complete overhaul of curricula and/or teaching methods.”
In the study, students switched back and forth from week to week between two different styles of homework. One style, which followed the standard practice that Baraniuk has used for years, consisted of one homework assignment per week, which was graded and returned the following week. The second style, which was called the “intervention,” incorporated three principles from cognitive science that have been shown to promote learning and increase long-term retention.
The principles were implemented in the following way:
1. Repeated retrieval practice – In addition to receiving the standard homework assignment, students were given follow-up problems on the same topic in two additional assignments that counted only toward their course participation grade.
2. Spacing — Rather than giving all the problem sets for a week’s lectures in one assignment, the researchers spaced the problems over three weeks of assignments.
3. Feedback — Rather than waiting one week to learn how they did, students received immediate feedback on intervention homework, and they were required to view the feedback to get credit for the assignment.
“Giving students multiple opportunities to practice retrieving and applying their knowledge on new problems is a very powerful way to promote learning, especially when this practice is spaced out over time,” said study co-author Elizabeth Marsh, associate professor of psychology and neuroscience at Duke. “Feedback also is critical to learning, and previous studies have shown that students will often skip looking at feedback.”
To alleviate any concerns that individual differences in ability might skew the study results, the researchers split the class into two groups and assigned each group standard homework and intervention homework during alternating weeks; in any given week, half of the students were assigned to the intervention and half to the standard practice homework.
The course covered 11 broad topics and approximately five core concepts per topic. Questions on the midterm and final exams covered specific concepts and topics, so by comparing how the two groups fared on those questions, the research team could compare whether students learned more doing one style of homework or the other. The research showed that students scored about 7 percent higher on the portions of the final exams that were taught with the intervention.
“In any experiment, it’s important to try to minimize the random sources of variance that might undermine the ability to detect whether an intervention had a significant effect,” said Baraniuk, the Victor E. Cameron Professor of Engineering in the Department of Electrical and Computer Engineering and director of Rice’s RDLS. “In signal processing, we refer to this as ‘noise,’ and classrooms are inherently ‘noisy’ because each student will engage with the material in many different ways.
“Besides homework assignments, students can also learn from lectures, do extra reading, go to review sessions and the like,” he said. “In general, the ‘noise’ created by all this outside learning would be expected to reduce or eliminate the kind of positive effects that we measured, so it’s noteworthy that we did not control for noise and still saw a significant effect.”
The study showed that the combination of repeated retrieval practice, spacing and immediate, required feedback had a powerful effect on student learning of complex engineering material.
Marsh said further research is needed to determine whether the results are broadly applicable across disciplines and grade levels, but the researchers made a conscious effort to design the study using broadly applicable methods.
“We aimed for changes that could easily be applied across disciplines and grade levels with minimal cost and disruption,” she said.
The software used in the test was developed in Baraniuk’s lab and is an academic research tool that has many of the same features as powerful learning aids that are currently on the market. These tools, which are sometimes called “digital tutors,” can be used as study aids or as platforms for administering homework and tests. In addition to collecting metrics that instructors can use to evaluate individual students or entire course sections, the most sophisticated tutors are “intelligent” in that they learn about each student and tailor instruction for each individual.
“We could have implemented these same principles in the classroom without technology, but the digital tutor made it much easier,” Butler said. “Moreover, technology has the potential to implement these principles in a more powerful way by providing personalized instruction to each student.”
