How Much is Lost? Measuring Long-Term Learning Using Multiple Choice Tests

Abstract

This work proposes a new approach for measuring long-term conceptual knowledge based on the after-instruction evolution of students´ answers to a research-based, multiple-choice, single-response test. The method allows for a quantitative determination of the fraction of students that, after instruction, attain long-lasting and temporary learnings, as well as those that did not learn. It also provides a plausible value of the experimental error. The method has been applied to analyze data obtained from a group comparison quasi-experimental design, in which two intact, equivalent high school classes have been subjected to two different instructional approaches. Conceptual knowledge of the subject, simple resistive electric circuits, was measured through the administration of the multiple-choice test DIRECT at three different times: before and immediately after instruction and one year later. Results indicate that the fraction of students achieving long-term learning is about four times larger in the group that followed active-learning activities, compared with the class that followed traditional instruction; drastically decreasing the no-learning group. The proposed method is relatively simple to implement and to interpret, providing more in-depth information, with higher accuracy and detail than the usual pre- and post-instruction data analysis. Some suggestions for complementary studies and to improve instruction are also given.