[Paper Review] Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling

Philip I. Pavlik, Luke G. Eglington+1 more|arXiv (Cornell University)|May 2, 2020

Intelligent Tutoring Systems and Adaptive LearningComputer Science61 references30 citations

TL;DR

This paper introduces Logistic Knowledge Tracing (LKT), a constrained framework that formalizes the specification of logistic regression models for learner modeling in adaptive learning systems. By using a symbolic notation system to represent diverse models—including variants of existing methods and novel configurations—LKT enables systematic comparison and improved generalizability without relying on student-specific fixed parameters, with empirical results showing no single model outperforms others across all datasets.

ABSTRACT

Adaptive learning technology solutions often use a learner model to trace learning and make pedagogical decisions. The present research introduces a formalized methodology for specifying learner models, Logistic Knowledge Tracing (LKT), that consolidates many extant learner modeling methods. The strength of LKT is the specification of a symbolic notation system for alternative logistic regression models that is powerful enough to specify many extant models in the literature and many new models. To demonstrate the generality of LKT, we fit 12 models, some variants of well-known models and some newly devised, to 6 learning technology datasets. The results indicated that no single learner model was best in all cases, further justifying a broad approach that considers multiple learner model features and the learning context. The models presented here avoid student-level fixed parameters to increase generalizability. We also introduce features to stand in for these intercepts. We argue that to be maximally applicable, a learner model needs to adapt to student differences, rather than needing to be pre-parameterized with the level of each student's ability.

Motivation & Objective

• To develop a unified, formalized framework for specifying logistic regression models in learner modeling.
• To improve model generalizability by avoiding student-level fixed parameters and instead using feature-based intercepts.
• To facilitate systematic comparison of multiple learner model configurations across diverse learning contexts.
• To accelerate research by enabling rapid prototyping and testing of logistic regression models via an R package and LearnSphere integration.
• To support the creation of more transparent, systematic, and extensible learner models for adaptive educational technology.

Proposed method

• LKT uses a symbolic notation system to represent logistic regression models, allowing flexible specification of features and interactions.
• The framework supports both fixed and random effects for knowledge components (KCs), with a focus on fixed effects for improved reliability.
• It enables the use of nonlinear and composite features (e.g., interactions between recency, prior performance, and forgetting terms) that are difficult to implement in standard logistic regression.
• LKT is implemented as an R package and integrated into the LearnSphere platform for use with DataShop datasets.
• The system supports model fitting using sparse matrix computation and optimized libraries (e.g., data.table, LibLineaR) for high performance.
• Future extensions include enhanced feature composition (e.g., automatic interaction terms) and integration of latency models for efficiency-based practice selection.

Experimental results

Research questions

• RQ1 Which combinations of model features yield the most accurate predictions of student performance across diverse learning tasks?
• RQ2 Can a unified symbolic framework improve the systematic development and comparison of learner models?
• RQ3 Does avoiding student-specific fixed parameters enhance model generalizability across new learners?
• RQ4 How do memory-based factors (e.g., recency, forgetting) compare to procedural/conceptual learning factors in predicting performance?
• RQ5 To what extent can feature engineering within LKT capture domain-specific learning patterns, such as category learning or perceptual-motor tasks?

Key findings

• No single learner model was optimal across all six datasets, highlighting the need for context-sensitive model selection.
• Models incorporating recency and forgetting effects performed best on memory-based tasks (MHE and Cloze), where memorization played a key role.
• The R-PFA-inspired 'propdec' feature, which tracks recent performance without explicit timing, performed comparably to models using actual time intervals.
• The LKT framework enabled efficient fitting of complex models using sparse computation, making previously infeasible analyses in base R now practical.
• The framework demonstrated strong empirical performance across diverse domains, including procedural, conceptual, and perceptual learning tasks.
• The authors observed that transfer between knowledge components could improve performance on tasks like Chinese tones, suggesting the value of modeling inter-KC relationships.