By Phil Hill
Update 10/25: Bumped comment from Darryl Yong, a member of the research team, into its own post here.
Update 10/26: We now have Rachel Levy and Nancy Lape (who was the researcher interviewed by USA Today) both agreeing with Darryl’s comments. That’s three of the four members of the research team. While I do not claim to understand how the reporter developed her story line (I have asked for comment), it is quite clear that the article does not represent the work of the Harvey Mudd research team and has a misleading headline and lede.
USA Today published an article today titled “‘Flipped classrooms’ may not have any impact on learning“, based on research from four Harvey Mudd professors. This research is backed by a “$199,544 grant from the National Science Foundation to study the effects of the flipped classroom on students’ learning”. This is newsworthy, right? A real research report with NSF funding finding no statistical difference in learning outcomes from flipped classroom seems to contradict much of the recent promise of ed tech.
Upon closer reading, however, there are some major problems with the story. Exaggerated claims by ed tech enthusiasts are not helpful, but neither are exaggerated claims by ed tech skeptics. We at e-Literate have been critical of both flavors (witness our analysis of San Jose State claims, Desire2Learn claims, and edX claims for examples of the former).
Let’s review today’s story as an example of the latter.
In a flipped classroom, students watch their professors’ lectures online before class, while spending class time working on hands-on, “real world” problems.
The potential of the model has many educators thrilled — it could be the end of vast lecture halls, students falling asleep and boring, monotone professors.
This is a decent summary, although I would argue there should not be a one-size-fits-all mentality. Flipped classrooms, even where successful, should not replace all lecture-based classes. But for national media, this is not a bad description.
But four professors at Harvey Mudd College in Claremont, Calif. who are studying the effectiveness of a flipped classroom have bad news for advocates of the trend: it might not make any difference.
This is the lede, and the claim that we should examine. What is the basis of their study?
Though their official research is just beginning, the professors flipped their STEM classrooms as a pilot during the 2012-2013 academic year and gathered some first impressions on the matter.
While [Professor Nancy] Lape stresses that their preliminary research is just that — preliminary — she says the benefits of flipping a classroom are dubious.
During this pilot, each professor taught two sections of the same course — one “flipped” and one traditional, using the same material as much as possible.
So there is no report yet, and the study is not complete. But they are ready to make conclusions? I’d be willing to bet that the vast majority of readers will remember the headline and lede and not the qualifier of “preliminary research”. There is no ability for people to study the team’s results given the nature of this article, so we are supposed to just trust the research team and the reporter.
The potential of flipped classrooms, or any redesign, is not based on just changing delivery. It often comes from changing or enhancing the learning content to fit the course redesign. This study seems to force-fit traditional material into a flipped classroom. While that might be appropriate for their classes, it is not the only option.
And making conclusions on eight classes – four traditional, four flipped – taught by the the same four professors? While that is great for self-reflection and experimentation, I’m glad they aren’t extrapolating their preliminary findings.
“I would say that the fact is that there is no statistical difference,” Lape says. “People are really gung ho about the (flipped) classroom, but there’s no real results.”
Wow. They have enough data to claim the fact of no statistical difference and no results on flipped classrooms.
Does the research team know about the work of the National Center of Academic Transformation or the Open Learning Initiative and their documented results on course redesign? There are other studies showing promising results from the flipped classroom concept, including the recent redesign at San Jose State University. None are fully conclusive for all cases, but to claim that there are “no real results” seems disingenuous.
Did I mention (thanks to reminder from @GlobalHigherEd) that Harvey Mudd has a 9:1 student to teacher ratio? That’s not exactly a good basis for extrapolating results to the broad usage of flipped classrooms. If all colleges had 9:1 ratios, it would be difficult to find any course redesign with improved results. Not impossible, but the bar would be set much higher.
Maybe the research team will be cautious about advising other faculty what to do, given their very unique experience at Harvey Mudd.
Professors, too, had to spend considerably more time making and editing the videos and crafting engaging, hands-on sessions for their classes, she says.
Given these drawbacks, the fact that the actual learning outcomes seemed unaffected by the switch suggested that it might not be worth the hassle, Lape says.
“(The professors’) lives might be easier and their students might be happier if they just do a traditional class,” she says.
Yikes. Those are some pretty audacious claims in national media for preliminary results from eight classes taught by the research team themselves at a school with 9:1 student to faculty ration. At the end the reporter adds some needed context.
Andrew Miller, an education consultant who teaches online classes for a variety of universities, agrees that benefits such as students’ ability to review material are promising, but says nothing will change if professors don’t handle the “flip” correctly.
For example, the newly freed-up class time can be daunting for professors, especially those who are particularly gifted at lecturing, he says.
Sometimes these professors aren’t able to come up with good hands-on activities and resort to filling the time with even more lecturing.
“If you’re not a good instructor, flipping the classroom won’t really ensure better learning,” he says. “If you aren’t doing something to fill that space, it won’t do you any good.”
And to be fair, the quoted professor at the end softens her tone.
Lape says she hopes those within academia take a more critical look at flipped classrooms.
“It’s a hot topic, and there are reasons why I think people believe it will be a good method,” Lape says. “But I would really put the call out to more people to really look at this.” [emphasis added]
That is a call that I support.
There could be an argument that this article is a case of a reporter trying to find a sensational topic from a nuanced report. But the real problems in this article seem to be direct quotes from one of the research professors, despite the qualifier of “preliminary”.
We should expect better from both ed tech enthusiasts and from ed tech skeptics, especially when lending the credibility of official research. I look forward to the full report from the Harvey Mudd researchers.