[Paper Review] Predicting Student Performance in Interactive Online Question Pools Using Mouse Interaction Features

Huan Wei, Haotian Li+3 more|arXiv (Cornell University)|Jan 9, 2020

Online Learning and Analytics38 references4 citations

TL;DR

This paper proposes a novel method to predict student performance in interactive online question pools by leveraging mouse interaction features—such as think time, first attempt, and drag-and-drop patterns—alongside a heterogeneous information network (HIN) to model similarity between questions. The approach significantly improves prediction accuracy over traditional statistical features across multiple machine learning models, demonstrating the value of fine-grained behavioral data in unstructured, non-curricular learning environments.

ABSTRACT

Modeling student learning and further predicting the performance is a well-established task in online learning and is crucial to personalized education by recommending different learning resources to different students based on their needs. Interactive online question pools (e.g., educational game platforms), an important component of online education, have become increasingly popular in recent years. However, most existing work on student performance prediction targets at online learning platforms with a well-structured curriculum, predefined question order and accurate knowledge tags provided by domain experts. It remains unclear how to conduct student performance prediction in interactive online question pools without such well-organized question orders or knowledge tags by experts. In this paper, we propose a novel approach to boost student performance prediction in interactive online question pools by further considering student interaction features and the similarity between questions. Specifically, we introduce new features (e.g., think time, first attempt, and first drag-and-drop) based on student mouse movement trajectories to delineate students' problem-solving details. In addition, heterogeneous information network is applied to integrating students' historical problem-solving information on similar questions, enhancing student performance predictions on a new question. We evaluate the proposed approach on the dataset from a real-world interactive question pool using four typical machine learning models.

Motivation & Objective

• To address the challenge of predicting student performance in interactive online question pools lacking predefined question orders or expert-annotated knowledge tags.
• To explore whether mouse interaction features can provide meaningful signals for performance prediction in unstructured, exploratory learning environments.
• To integrate historical problem-solving patterns on similar questions using a heterogeneous information network (HIN) to improve prediction robustness.
• To evaluate the effectiveness and consistency of the proposed method across diverse machine learning models and question classes.

Proposed method

• Extract mouse movement trajectory features including think time, first attempt, and first drag-and-drop events from logged interaction data.
• Use a sliding window and density threshold to detect change points in mouse drag events, generating a drag event histogram for each question.
• Construct a heterogeneous information network (HIN) that links students, questions, and interaction features to model relationships across similar questions.
• Apply the HIN to aggregate historical performance data from semantically similar questions, enhancing predictions for new questions.
• Combine the extracted interaction features with traditional statistical features (e.g., average scores) and feed them into standard machine learning models (e.g., GBDT, SVM, LR, RF).
• Use cosine similarity to identify questions with similar interaction patterns, selecting only those with a similarity score ≥0.7 for HIN propagation.

Experimental results

Research questions

• RQ1 Can mouse interaction features such as think time and drag-and-drop patterns improve student performance prediction in interactive online question pools?
• RQ2 How effective is the integration of historical performance on similar questions via a heterogeneous information network (HIN) in boosting prediction accuracy?
• RQ3 Does the proposed method outperform traditional statistical baselines across diverse machine learning models and question classes?
• RQ4 How consistent is the performance of the proposed method across different models and question types?

Key findings

• The proposed method achieved significantly higher prediction accuracy than the baseline model using only statistical features (e.g., average scores) across all tested models.
• The inclusion of mouse interaction features contributed substantially to performance, with the GBDT model showing the most notable improvement, indicating the features' strong predictive power.
• The heterogeneous information network (HIN) effectively leveraged historical data from similar questions, enhancing predictions for new questions by capturing transferable learning patterns.
• The method demonstrated consistent performance gains across different machine learning models, including GBDT, SVM, logistic regression, and random forest, across various question classes.
• The results confirm that interaction-based features are more informative than statistical features alone in environments without structured curricula or precise knowledge tags.