We conducted two AB experiments (treatment vs. control) in a massive open online course. The first experiment evaluates deliberate practice activities (DPAs) for developing problem solving expertise as measured by traditional physics problems. We find that a more interactive drag-and-drop format of DPA generates quicker learning than a multiple choice format but DPAs do not improve performance on solving traditional physics problems more than normal homework practice. The second experiment shows that a different video shooting setting can improve the fluency of the instructor which in turn improves the engagement of the students although it has no significant impact on the learning outcomes. These two cases demonstrate the potential of MOOC AB experiments as an open-ended research tool but also reveal limitations. We discuss the three most important challenges: wide student distribution, “open-book” nature of assessments, and large quantity and variety of data. We suggest possible methods to cope with those.
|Journal||Research and Practice in Technology Enhanced Learning|
|State||Published - 1 Dec 2016|
Bibliographical noteFunding Information:
We would like to thank Hallie Voulgaris for serving as teaching assistant in videos used in experiment 2, and John Daries from MITx for providing the MOOC log data. We thank MIT and Google for supporting this research.
© 2016, The Author(s).
- Cognitive Load
- Deliberate Practice
- Extraneous Cognitive Load
- Multiple Choice Format
- Traditional Problem