[论文解读] Revolutionising Distance Learning: A Comparative Study of Learning Progress with AI-Driven Tutoring

Moritz Möller, G K Nirmal+5 more|arXiv (Cornell University)|Feb 21, 2024

Engineering Education and Technology被引用 13

一句话总结

研究表明，AI 辅导助手（Syntea）在远程学习项目中加快了大学学习进程，在上线后三个月内，基于对40余门课程的大规模分析，学习时间减少约27%。

ABSTRACT

Generative AI is expected to have a vast, positive impact on education; however, at present, this potential has not yet been demonstrated at scale at university level. In this study, we present first evidence that generative AI can increase the speed of learning substantially in university students. We tested whether using the AI-powered teaching assistant Syntea affected the speed of learning of hundreds of distance learning students across more than 40 courses at the IU International University of Applied Sciences. Our analysis suggests that using Syntea reduced their study time substantially--by about 27\% on average--in the third month after the release of Syntea. Taken together, the magnitude of the effect and the scalability of the approach implicate generative AI as a key lever to significantly improve and accelerate learning by personalisation.

研究动机与目标

• 激励并量化生成式人工智能辅导如何加速远程学习项目中的大学学习。
• 评估 Syntea 的考试训练功能对学习进展随时间的影响。
• 评估效应对人群变化和潜在混杂因素的稳健性。
• 检查对通过率和成绩等学习结果的影响，以评估潜在权衡。

提出的方法

• 在数百名远程学习学生中使用处理组（Syntea 考试训练器用户）对照组（非用户）设计。
• 在上线前24个月至上线后3个月的时间线上，使用60天滑动窗口计算学习进展（每个学生每月通过的考试数）。
• 在治疗组和对照组之间执行基线校正、逐月相对差异分析。
• 进行前后对比并使用t检验评估观察效应的统计显著性（包括稳健性检验）。
• 补充分析包括通过率和分数统计，以评估对结果的潜在副作用。

实验结果

研究问题

• RQ1 Does the AI tutoring assistant Syntea accelerate learning progress in university distance programs?
• RQ2 How large is the acceleration effect and how does it evolve over time after rollout?
• RQ3 Is the observed effect robust to changes in student population and seasonal dynamics?
• RQ4 Do AI tutoring interventions impact pass rates or average grades?

主要发现

• Syntea users learned faster than the average distance learner even before rollout (about 24.1% higher progress).
• Post-rollout, the gap widened with relative differences of 27.9% (Month 1), 39.0% (Month 2), and 69.9% (Month 3) in study progression.
• Statistical tests show significant improvements post-release: p = .031 (Month 2) and p < .001 (Month 3); pre-release difference was not statistically significant.
• The study translates the progress gains into study-time reductions, illustrating a ~27% decrease in average study duration (months/exam) for Syntea users by Month 3.
• Pass rates remained high and stable (~0.98) with no significant adverse effects; average grades showed a small, statistically significant but small decline (-3.4 percentage points) in the treatment group by Month 3.
• An estimated acceleration of learning by roughly 46 percentage points relative to the control group is reported, supporting the potential of AI tutors as a scalable lever for faster learning.