Problems with study that claims groups are better than lectures for learning

Last month the NY Times reported on a study that showed interactive group instruction is superior to traditional lectures.

In one of the initiative’s most visible studies, Dr. Wieman’s team reports that students in an introductory college physics course did especially well on an exam after attending experimental, collaborative classes during the 12th week of the course. By contrast, students taking the same course from another instructor — who did not use the experimental approach and continued with lectures as usual — scored much lower on the same exam.

Not so fast.

Yet experts who reviewed the new report cautioned that it was not convincing enough to change teaching. The study has a variety of limitations, they said, some because of the difficulty of doing research in the dude-I-slept-through-class world of the freshman year of college, and others because of the study’s design. “The whole issue of how to draw on basic science and apply it in classrooms is a whole lot more complicated than they’re letting on,” said Daniel Willingham, a psychology professor at the University of Virginia.
Dr. Willingham said that, among other concerns, the study was not controlled enough to tell which of the changes in teaching might have accounted for the difference in students’ scores. 

Here are Dr. Willingham's candid comments from his Facebook page.

There were a lot of problems with this study. The two methods compared were each tested in just ONE classroom--so no way of knowing whether the observed effects were just due to the teacher. The "group learning" condition was taught by a new teacher--the "lecture" was the same professor as had been teaching all semester. The critical test was opt-in, and lots of students decided not to take it--and the proportions were unequal across conditions. The study was a mess. I can't imagine why Science published it....
Studies like this are fine for what they are--they are really more pilot studies, or they could be useful as qualitative research. (I think qualitative research *is* really quite useful.) But it didn't have the strengths of qualitative research and was pitched as a quantitative study.

Other articles about this study:
Science
The Economist
The Chronicle of Higher Ed