[论文解读] Deep Knowledge Tracing with Convolutions

Shanghui Yang, Mengxia Zhu+2 more|arXiv (Cornell University)|Jul 26, 2020

Intelligent Tutoring Systems and Adaptive Learning参考文献 27被引用 6

一句话总结

本文提出卷积知识追踪（CKT），一种深度学习模型，通过使用三维卷积捕捉学生近期答题的短期影响，从而更有效地建模遗忘曲线。CKT在ASSISTments2009数据集上相较于DKT和DKVMN分别实现AUC提升1.55和2.03，在ASSISTments2015数据集上分别实现AUC提升1.01和1.96，达到当前最优性能。

ABSTRACT

Knowledge tracing (KT) has recently been an active research area of computational pedagogy. The task is to model students mastery level of knowledge based on their responses to the questions in the past, as well as predict the probabilities that they correctly answer subsequent questions in the future. A good KT model can not only make students timely aware of their knowledge states, but also help teachers develop better personalized teaching plans for students. KT tasks were historically solved using statistical modeling methods such as Bayesian inference and factor analysis, but recent advances in deep learning have led to the successive proposals that leverage deep neural networks, including long short-term memory networks, memory-augmented networks and self-attention networks. While those deep models demonstrate superior performance over the traditional approaches, they all neglect more or less the impact on knowledge states of the most recent questions answered by students. The forgetting curve theory states that human memory retention declines over time, therefore knowledge states should be mostly affected by the recent questions. Based on this observation, we propose a Convolutional Knowledge Tracing (CKT) model in this paper. In addition to modeling the long-term effect of the entire question-answer sequence, CKT also strengthens the short-term effect of recent questions using 3D convolutions, thereby effectively modeling the forgetting curve in the learning process. Extensive experiments show that CKT achieves the new state-of-the-art in predicting students performance compared with existing models. Using CKT, we gain 1.55 and 2.03 improvements in terms of AUC over DKT and DKVMN respectively, on the ASSISTments2009 dataset. And on the ASSISTments2015 dataset, the corresponding improvements are 1.01 and 1.96 respectively.

研究动机与目标

• 为解决现有深度学习模型在知识追踪中的局限性，即往往低估近期学生答题的影响。
• 通过强调近期问题的影响，将遗忘曲线理论融入基于深度学习的知识追踪。
• 通过同时建模长期与短期知识状态动态，提升预测未来学生表现的准确性。
• 开发一种新型架构，结合序列建模与使用三维卷积提取局部时间模式。

提出的方法

• CKT模型使用三维卷积层处理学生答题序列，捕捉跨多个问题与知识概念的时间依赖性。
• 通过三维卷积提取正确/错误答题序列中的局部模式，突出最近的答题表现以建模短期知识保持。
• 模型整合来自循环网络或全连接网络的长期序列表征，以及来自三维卷积的短期模式，以优化知识状态估计。
• 最终的预测头利用融合表示估计学生正确回答下一题的概率。
• 该架构通过在学生答题序列上使用二元交叉熵损失进行端到端训练。
• 使用三维卷积使模型能够学习分层时间模式，其中高层滤波器聚焦于近期答题趋势。

实验结果

研究问题

• RQ1 建模近期学生答题的短期影响是否能提升知识追踪性能？
• RQ2 在深度学习模型中引入遗忘曲线理论在多大程度上能提升知识追踪的预测准确性？
• RQ3 所提出的三维卷积机制与标准RNN及注意力机制模型相比，在捕捉学习时间动态方面表现如何？
• RQ4 CKT模型是否能在不同教育数据集和学生答题模式下实现泛化？

主要发现

• 在ASSISTments2009数据集上，CKT相较于DKT模型AUC绝对提升1.55。
• 在同一数据集上，CKT相较于DKVMN模型AUC提升2.03。
• 在ASSISTments2015数据集上，CKT相较于DKT模型AUC提升1.01，相较于DKVMN模型AUC提升1.96。
• 性能提升表明，通过三维卷积建模近期答题能有效捕捉学生学习中的遗忘曲线。
• 结果证实，短期答题模式显著影响知识状态估计，应在知识追踪模型中优先考虑。
• CKT在两个基准数据集上均建立了知识追踪的新SOTA性能。