[Paper Review] Analyzing Student Strategies In Blended Courses Using Clickstream Data

Nil-Jana Akpinar, Aaditya Ramdas+1 more|arXiv (Cornell University)|May 31, 2020

Online Learning and Analytics26 references4 citations

TL;DR

This paper proposes a hybrid NLP approach to analyze fine-grained clickstream data from a blended college course, revealing distinct student strategies for homework and exam preparation. It demonstrates that features like study session count, click frequency, and attendance predict performance reliably, even in low-data settings typical of blended courses.

ABSTRACT

Educational software data promises unique insights into students' study behaviors and drivers of success. While much work has been dedicated to performance prediction in massive open online courses, it is unclear if the same methods can be applied to blended courses and a deeper understanding of student strategies is often missing. We use pattern mining and models borrowed from Natural Language Processing (NLP) to understand student interactions and extract frequent strategies from a blended college course. Fine-grained clickstream data is collected through Diderot, a non-commercial educational support system that spans a wide range of functionalities. We find that interaction patterns differ considerably based on the assessment type students are preparing for, and many of the extracted features can be used for reliable performance prediction. Our results suggest that the proposed hybrid NLP methods can provide valuable insights even in the low-data setting of blended courses given enough data granularity.

Motivation & Objective

• To understand how students interact with educational software in blended courses, particularly their learning strategies.
• To address the gap in performance prediction for blended courses, where data is sparser and less structured than in MOOCs.
• To develop and validate a novel method for extracting meaningful, interpretable student strategies from low-data clickstream logs.
• To investigate how strategies differ based on assessment type (homework vs. exam) and their predictive power for academic outcomes.
• To provide actionable insights for instructors and early warning system design using context-aware, behavior-based features.

Proposed method

• Preprocess raw clickstream data from Diderot, a non-commercial educational support system, into action sequences representing student interactions.
• Apply topic modeling (e.g., Latent Dirichlet Allocation) to identify recurring patterns of student behavior as 'strategies'.
• Use skip-gram neural network models to learn contextual embeddings of student actions, capturing sequential and semantic relationships.
• Combine pattern mining with NLP techniques to extract frequent, meaningful strategies from fine-grained interaction logs.
• Integrate derived features—such as study session count, click frequency, and attendance—into tree-based ensemble models for performance prediction.
• Leverage partial dependence plots and feature importance scores to interpret model outputs and enhance model explainability.

Experimental results

Research questions

• RQ1 How do students interact with course material, and what are the frequent strategies they employ?
• RQ2 How do students’ strategies differ when preparing for homework versus exams?
• RQ3 Are student interaction patterns and strategies predictive of academic performance outcomes?
• RQ4 Can hybrid NLP methods effectively extract interpretable strategies from low-data, blended course clickstream data?
• RQ5 Which behavioral features are most indicative of high performance in blended learning environments?

Key findings

• Students exhibit increased activity in the days leading up to deadlines and exams, with exam preparation involving more review of lecture notes than homework solving.
• Students ask more questions about homework than about other materials like lecture notes or practice exams, indicating targeted engagement with assignment content.
• Students who attend lectures and recitations score significantly higher on average—76.58% vs. 55.86% for homework—demonstrating attendance as a strong predictor of performance.
• The number of study sessions, total clicks, and engagement with homework-specific content are strong predictors of homework grades in the model.
• Topic models like LDA recover similar strategies as traditional pattern mining, while skip-gram models provide valuable context-aware representations even with moderate data.
• Tree-based ensemble models offer high interpretability through feature importance and partial dependence plots, making them suitable for high-stakes early warning systems.