[论文解读] BKT-LSTM: Efficient Student Modeling for knowledge tracing and student performance prediction

Sein Minn|arXiv (Cornell University)|Dec 22, 2020

Intelligent Tutoring Systems and Adaptive Learning被引用 7

一句话总结

该论文提出BKT-LSTM，一种知识追踪模型，通过将贝叶斯知识追踪（BKT）用于技能掌握建模、k-means聚类用于学生能力特征（学习迁移）以及问题难度，整合进基于LSTM的框架中。该模型通过利用具有心理可解释性的特征而非原始的二值交互序列，在多个数据集上实现了优于最先进模型的学生表现预测性能。

ABSTRACT

Recently, we have seen a rapid rise in usage of online educational platforms. The personalized education became crucially important in future learning environments. Knowledge tracing (KT) refers to the detection of students' knowledge states and predict future performance given their past outcomes for providing adaptive solution to Intelligent Tutoring Systems (ITS). Bayesian Knowledge Tracing (BKT) is a model to capture mastery level of each skill with psychologically meaningful parameters and widely used in successful tutoring systems. However, it is unable to detect learning transfer across skills because each skill model is learned independently and shows lower efficiency in student performance prediction. While recent KT models based on deep neural networks shows impressive predictive power but it came with a price. Ten of thousands of parameters in neural networks are unable to provide psychologically meaningful interpretation that reflect to cognitive theory. In this paper, we proposed an efficient student model called BKT-LSTM. It contains three meaningful components: individual \ extit{skill mastery} assessed by BKT, \ extit{ability profile} (learning transfer across skills) detected by k-means clustering and \ extit{problem difficulty}. All these components are taken into account in student's future performance prediction by leveraging predictive power of LSTM. BKT-LSTM outperforms state-of-the-art student models in student's performance prediction by considering these meaningful features instead of using binary values of student's past interaction in DKT. We also conduct ablation studies on each of BKT-LSTM model components to examine their value and each component shows significant contribution in student's performance prediction. Thus, it has potential for providing adaptive and personalized instruction in real-world educational systems.

研究动机与目标

• 为解决BKT的局限性，即独立建模各技能，无法捕捉技能间的知识迁移。
• 克服DKT等深度学习模型的黑箱特性，尽管其预测性能强大，但缺乏心理学可解释性。
• 开发一种混合模型，在保持认知可解释性的同时，提升知识追踪中的预测准确性。
• 通过实证验证各组成部分——技能掌握度、能力特征和问题难度——在学生表现预测中的贡献。

提出的方法

• BKT-LSTM使用贝叶斯知识追踪来估计每个技能的学生掌握概率。
• 通过k-means聚类对学生交互序列进行处理，以提取能力特征，反映技能间的知识迁移。
• 将问题难度作为输入特征显式建模，反映题目层面的难度挑战。
• LSTM处理技能掌握度、能力特征和问题难度的序列，以预测下一次答题的正确概率。
• 模型通过标准反向传播进行端到端训练，损失函数通过二元交叉熵最小化。
• 通过消融研究隔离各组件对预测性能的贡献。

实验结果

研究问题

• RQ1 将基于BKT的技能掌握度、学生能力特征和问题难度整合进LSTM框架，是否能提升相较于仅使用二值交互序列模型的性能？
• RQ2 各组件——技能掌握度、能力特征和问题难度——在知识追踪预测准确性中的具体贡献如何？
• RQ3 与纯数据驱动的深度学习模型（如DKT）相比，引入可解释的认知特征是否能提升模型性能？
• RQ4 当数据可用性不同时（如问题难度信息是否存在），该模型在多样化数据集上的表现如何？
• RQ5 能力特征组件在多大程度上反映了长期学习迁移，并在个体技能掌握之外提升预测性能？

主要发现

• BKT-LSTM在ASSISTments09数据集上AUC达到0.802，ASSISTments14上为0.707，Algebra数据集上为0.851，优于DKT及其他最先进模型。
• 消融研究显示，包含问题难度的模型相比无此组件的模型性能提升8–10%，表明其为最具影响力的组件。
• BKT-LSTM-4（含所有组件）在ASSISTments09上RMSE最低，为0.409；ASSISTments14上为0.420；Algebra上为0.353，表明其回归性能更优。
• 能力特征组件贡献显著，相比无此组件的模型性能提升2–4%，尤其在长期学习预测中表现更优。
• 与BKT相比，BKT-LSTM性能显著提升，即使仅包含掌握度的BKT-LSTM-1也已超越BKT的性能。
• 在ASSISTments14数据集上，由于缺乏问题难度信息，所有模型表现相近，证实当问题难度信息可用时，其为关键影响因素。