Researching for better instructional methods using AB experiments in MOOCs: results and challenges

Zhongzhou Chen, Christopher Chudzicki, Daniel Palumbo, Giora Alexandron, Youn Jeng Choi, Qian Zhou, David E. Pritchard

Research output: Contribution to journalArticlepeer-review

14 Scopus citations


We conducted two AB experiments (treatment vs. control) in a massive open online course. The first experiment evaluates deliberate practice activities (DPAs) for developing problem solving expertise as measured by traditional physics problems. We find that a more interactive drag-and-drop format of DPA generates quicker learning than a multiple choice format but DPAs do not improve performance on solving traditional physics problems more than normal homework practice. The second experiment shows that a different video shooting setting can improve the fluency of the instructor which in turn improves the engagement of the students although it has no significant impact on the learning outcomes. These two cases demonstrate the potential of MOOC AB experiments as an open-ended research tool but also reveal limitations. We discuss the three most important challenges: wide student distribution, “open-book” nature of assessments, and large quantity and variety of data. We suggest possible methods to cope with those.

Original languageEnglish
Article number9
JournalResearch and Practice in Technology Enhanced Learning
Issue number1
StatePublished - 1 Dec 2016

Bibliographical note

Funding Information:
We would like to thank Hallie Voulgaris for serving as teaching assistant in videos used in experiment 2, and John Daries from MITx for providing the MOOC log data. We thank MIT and Google for supporting this research.

Publisher Copyright:
© 2016, The Author(s).


  • Cognitive Load
  • Deliberate Practice
  • Extraneous Cognitive Load
  • Multiple Choice Format
  • Traditional Problem


Dive into the research topics of 'Researching for better instructional methods using AB experiments in MOOCs: results and challenges'. Together they form a unique fingerprint.

Cite this