[Paper Review] GritNet: Student Performance Prediction with Deep Learning

Byung‐Hak Kim, Ethan Vizitei+1 more|arXiv (Cornell University)|Apr 19, 2018

Online Learning and Analytics7 references87 citations

TL;DR

GritNet treats student performance prediction as sequential event prediction and uses a BLSTM with an embedding and a GMP layer to predict future graduation likelihood from raw time-stamped events, outperforming a logistic regression baseline especially in early weeks.

ABSTRACT

Student performance prediction - where a machine forecasts the future performance of students as they interact with online coursework - is a challenging problem. Reliable early-stage predictions of a student's future performance could be critical to facilitate timely educational interventions during a course. However, very few prior studies have explored this problem from a deep learning perspective. In this paper, we recast the student performance prediction problem as a sequential event prediction problem and propose a new deep learning based algorithm, termed GritNet, which builds upon the bidirectional long short term memory (BLSTM). Our results, from real Udacity students' graduation predictions, show that the GritNet not only consistently outperforms the standard logistic-regression based method, but that improvements are substantially pronounced in the first few weeks when accurate predictions are most challenging.

Motivation & Objective

• Motivate accurate, early predictions of student graduation to enable timely interventions in online courses.
• Recast student performance prediction as a sequential event prediction problem and apply deep learning without heavy feature engineering.
• Demonstrate that GritNet improves graduation prediction over a logistic regression baseline, particularly in the first weeks of a course.
• Evaluate GritNet on real Udacity Nanodegree data to assess early predictive power and robustness to imbalanced data.

Proposed method

• Encode time-stamped student actions as a fixed-length input via one-hot encoding and discretized inter-event times.
• Embed events with a learnable embedding matrix to produce dense representations.
• Model sequence with bidirectional LSTM layers to capture temporal context.
• Apply a global max pooling (GMP) layer to focus on discriminative parts of the sequence before a dense output.
• Train with negative log-likelihood (binary cross-entropy) and use dropout for regularization.
• Compare against a logistic-regression baseline using bag-of-words features and perform 5-fold cross-validation.

Experimental results

Research questions

• RQ1 Can GritNet predict final graduation from partial student event histories more accurately than a logistic regression baseline?
• RQ2 Does the GMP layer improve prediction under imbalanced data and with variable-length event sequences?
• RQ3 How early (in weeks) can GritNet provide superior graduation predictions relative to baselines?
• RQ4 How does GritNet perform across different Udacity Nanodegree datasets with varying curricula and graduation rates?

Key findings

• GritNet achieves higher prediction accuracy than the logistic regression baseline across weeks on both ND-A and ND-B datasets.
• GritNet provides more than 5.0 percentage points absolute improvement in AUC at early weeks (week 1 for ND-A; week 3 for ND-B).
• On ND-B, the baseline needs eight weeks of data to reach the same performance that GritNet attains with three weeks of data.
• GMP helps address data imbalance and boosts predictive power compared to a vanilla BLSTM without GMP.
• GritNet demonstrates strong early-stage predictive power, enabling faster feedback loops for interventions.