[论文解读] Next-Term Student Performance Prediction: A Recommender Systems Approach

Mack Sweeney, Huzefa Rangwala+2 more|arXiv (Cornell University)|Apr 7, 2016

Online Learning and Analytics参考文献 28被引用 105

一句话总结

论文通过将内容特征与协同过滤相结合，使用FM、RF和PMLR模型，提出一个面向传统大学的下一学期成绩预测系统，并引入一种新颖的特征重要性度量以提高准确性和可解释性。

ABSTRACT

An enduring issue in higher education is student retention to successful graduation. National statistics indicate that most higher education institutions have four-year degree completion rates around 50 percent, or just half of their student populations. While there are prediction models which illuminate what factors assist with college student success, interventions that support course selections on a semester-to-semester basis have yet to be deeply understood. To further this goal, we develop a system to predict students' grades in the courses they will enroll in during the next enrollment term by learning patterns from historical transcript data coupled with additional information about students, courses and the instructors teaching them. We explore a variety of classic and state-of-the-art techniques which have proven effective for recommendation tasks in the e-commerce domain. In our experiments, Factorization Machines (FM), Random Forests (RF), and the Personalized Multi-Linear Regression model achieve the lowest prediction error. Application of a novel feature selection technique is key to the predictive success and interpretability of the FM. By comparing feature importance across populations and across models, we uncover strong connections between instructor characteristics and student performance. We also discover key differences between transfer and non-transfer students. Ultimately we find that a hybrid FM-RF method can be used to accurately predict grades for both new and returning students taking both new and existing courses. Application of these techniques holds promise for student degree planning, instructor interventions, and personalized advising, all of which could improve retention and academic performance.

研究动机与目标

• 在高等教育中激发并解决长期存在的学生留存挑战。
• 使用历史成绩单以及学生、课程和教师特征来预测下一入学学期的学生成绩。
• 评估混合内容基和协同过滤模型以处理冷启动和非冷启动场景。
• 分析特征重要性，揭示影响表现的教师和转学学生因素。
• 提出一种方法以提升解释性和对学位规划与咨询的预测准确性。

提出的方法

• 将下一学期成绩预测形式化为一个在稀疏的学生-课程矩阵上具有丰富内容特征的回归任务。
• 通过简单基线、矩阵分解（SVD、SVD-kNN）以及通过 FM 的混合模型进行探索。
• 通过 FM 将内容特征合入，以应对冷启动并学习二阶交互。
• 评估回归模型包括 RF、SGD 回归、kNN 和 PMLR；实现用于 FM 的新颖 MADImp 特征重要性度量。
• 提出一种混合 FM-RF 方法，利用两者的优点并缓解冷启动问题。
• 为避免数据泄漏，对每个学术学期训练一个模型，并预测本学期的成绩。

实验结果

研究问题

• RQ1 使用历史成绩单并增添学生、课程和教师特征，下一学期成绩可以预测到什么程度？
• RQ2 将内容特征与协同过滤方法结合，对冷启动和非冷启动的二元组的预测价值是多少？
• RQ3 哪些模型（FM、RF、PMLR 等）在下一学期成绩预测中产生最低的预测误差？
• RQ4 是否有新颖的特征重要性度量能提升可解释性并指导模型选择？
• RQ5 混合 FM-RF 方法在不同学生子群体（如转学与非转学学生）上是否优于单一模型？

主要发现

• 在测试的模型中，因子分解机（FM）、随机森林（RF）和个性化多线性回归（PMLR）实现了最低的预测误差。
• 通过 FM 将内容特征引入可通过学习到的二阶交互提升预测准确性和可解释性。
• 一种新颖的特征重要性度量（MADImp）提升了 FM 结果的可解释性，并有助于识别影响特征。
• 混合 FM-RF 方法通过结合优势并缓解冷启动问题，优于单一模型。
• 特征重要性分析显示教师特征与学生表现之间存在显著联系，转学与非转学学生之间也有差异。