[Paper Review] EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction

Qi Liu, Zhenya Huang+5 more|arXiv (Cornell University)|Jun 7, 2019

Online Learning and Analytics53 references52 citations

TL;DR

The paper introduces EERNN and EKT frameworks that predict student performance by jointly leveraging exercise content and student records, with EKT providing interpretable tracking of knowledge acquisition across multiple concepts.

ABSTRACT

For offering proactive services to students in intelligent education, one of the fundamental tasks is predicting their performance (e.g., scores) on future exercises, where it is necessary to track each student's knowledge acquisition during her exercising activities. However, existing approaches can only exploit the exercising records of students, and the problem of extracting rich information existed in the exercise's materials (e.g., knowledge concepts, exercise content) to achieve both precise predictions of student performance and interpretable analysis of knowledge acquisition remains underexplored. In this paper, we present a holistic study of student performance prediction. To directly achieve the primary goal of prediction, we first propose a general Exercise-Enhanced Recurrent Neural Network (EERNN) framework by exploring both student's records and the exercise contents. In EERNN, we simply summarize each student's state into an integrated vector and trace it with a recurrent neural network, where we design a bidirectional LSTM to learn the encoding of each exercise's content. For making predictions, we propose two implementations under EERNN with different strategies, i.e., EERNNM with Markov property and EERNNA with Attention mechanism. Then, to explicitly track student's knowledge acquisition on multiple knowledge concepts, we extend EERNN to an explainable Exercise-aware Knowledge Tracing (EKT) by incorporating the knowledge concept effects, where the student's integrated state vector is extended to a knowledge state matrix. In EKT, we further develop a memory network for quantifying how much each exercise can affect the mastery of students on concepts during the exercising process. Finally, we conduct extensive experiments on large-scale real-world data. The results demonstrate the prediction effectiveness of two frameworks as well as the superior interpretability of EKT.

Motivation & Objective

• Motivate accurate prediction of future student performance while tracking knowledge acquisition over time.
• Combine exercise records with textual content of exercises to reduce information loss.
• Provide interpretable tracking of mastery over multiple knowledge concepts.
• Address cold-start scenarios through exercise-content and concept correlations.
• Offer two prediction strategies and extend to an explainable knowledge tracing framework.

Proposed method

• Propose EERNN to encode exercise content with a bidirectional LSTM and model student state with an RNN/LSTM that integrates exercise embeddings and responses.
• Implement two EERNN prediction strategies: EERNNM with Markov property and EERNNA with an attention mechanism over historical states.
• Extend EERNN to EKT by introducing a knowledge state matrix representing mastery over K knowledge concepts.
• Introduce a Knowledge Embedding module with a memory network to quantify how each exercise impacts each knowledge concept.
• Update concept-specific knowledge states via per-concept LSTMs conditioned on concept-aware inputs.
• Provide EKTM (Markov) and EKTA (Attention) variants for prediction within the EKT framework.

Experimental results

Research questions

• RQ1 How can exercise content be effectively incorporated into knowledge tracing to improve prediction of future performance?
• RQ2 Can predictions survive cold-start scenarios by leveraging exercise content and concept correlations?
• RQ3 Does explicit tracking of mastery over multiple knowledge concepts improve interpretability without sacrificing accuracy?
• RQ4 What is the impact of attention over historical exercises on predicting future performance?
• RQ5 How does the proposed memory-based knowledge embedding quantify cross-concept influences of exercises?

Key findings

• EERNN and EKT frameworks effectively predict student performance on future exercises.
• EKT provides superior interpretability by tracking mastery across explicit knowledge concepts.
• Attention-based EERNNA improves prediction by focusing on relevant historical states.
• Knowledge embedding captures correlations among concepts and aids in handling cold-start situations.
• Experimentation on large-scale real-world data demonstrates effectiveness in both general and cold-start scenarios.