The Suitability of External Control-Groups for Empirical Control Purposes: a Cautionary Story in Science Education Research

Abstract

This article deals with a specific effect in one external control group incorporated to account for
any pretest bias in a more comprehensive cognitive achievement study in a gene technology lab
(as part of a modified Solomon’s four-group plan). We monitored 12th graders (N = 117) in two
external groups without any intervention: a one-test group (n = 55) and a three-test group (n =
62). Both samples participated in identical tests which quantified the relevant knowledge of the
lesson unit applied in the main study. The three-test group yielded an unexpected increase in
achievement scores. Subsequent analysis revealed two subsamples: one with no changes, the
other with an increase (although without an intervention took place). A likely reason for the latter
situation may lie in the role of the teacher(s) involved who might have wish to avoid potential
negative results in his/her class. Consequently, we recommend the application of a modified
Solomon’s four group plan in science education research in order to prevent the influence of
teacher intervention in future empirical analyses.