[Paper Review] Getting too personal(ized): The importance of feature choice in online adaptive algorithms

Zhaobin Li, Luna Yee+4 more|arXiv (Cornell University)|Sep 6, 2023

Advanced Bandit Algorithms ResearchDecision Sciences22 references7 citations

TL;DR

The paper investigates how including student characteristics in contextual multi-armed bandit (MAB) personalization can help or hinder online educational adaptive systems, showing benefits only when characteristics truly influence which version is best, and risks of bias when they do not.

ABSTRACT

Digital educational technologies offer the potential to customize students' experiences and learn what works for which students, enhancing the technology as more students interact with it. We consider whether and when attempting to discover how to personalize has a cost, such as if the adaptation to personal information can delay the adoption of policies that benefit all students. We explore these issues in the context of using multi-armed bandit (MAB) algorithms to learn a policy for what version of an educational technology to present to each student, varying the relation between student characteristics and outcomes and also whether the algorithm is aware of these characteristics. Through simulations, we demonstrate that the inclusion of student characteristics for personalization can be beneficial when those characteristics are needed to learn the optimal action. In other scenarios, this inclusion decreases performance of the bandit algorithm. Moreover, including unneeded student characteristics can systematically disadvantage students with less common values for these characteristics. Our simulations do however suggest that real-time personalization will be helpful in particular real-world scenarios, and we illustrate this through case studies using existing experimental results in ASSISTments. Overall, our simulations show that adaptive personalization in educational technologies can be a double-edged sword: real-time adaptation improves student experiences in some contexts, but the slower adaptation and potentially discriminatory results mean that a more personalized model is not always beneficial.

Motivation & Objective

• Assess how personalization through contextual MABs affects student outcomes in online educational technologies.
• Evaluate when including student characteristics improves or degrades performance under different outcome models.
• Investigate risks of bias and inequity arising from uneven distributions of characteristics.
• Bridge simulation results with real-world data to discuss practical implications for educational design.

Proposed method

• Use contextual Thompson sampling with a regularized Bayesian logistic regression to model reward probabilities given features.
• Simulate three outcome-generating models: Baseline, Universal optimal action, and Personalized optimal action.
• Vary the number of contextual variables from 1 to up to 10 to assess learning and regret.
• Examine horizon effects with class sizes of 50, 250, and 1000 students across 1000 trials each.
• Analyze performance with ANCOVA and report effect sizes and confidence intervals.

Figure 1: Swarm plots for the proportion of optimal actions for the two bandit types. Each point represents results from one trial with 250 students. For the universal optimal action, all scenarios show similar results; hence only scenario (1) is shown. The decreased performance of the contextual ba

Experimental results

Research questions

• RQ1 Under what conditions does including student characteristics in contextual MABs improve or impair learning outcomes?
• RQ2 How does the number of contextual features influence exploration, learning speed, and equity among student subgroups?
• RQ3 When does personalization yield the most benefit versus potential harm or discriminatory effects?
• RQ4 How do real-world distributions of characteristics affect the benefits or drawbacks of personalization?
• RQ5 What guidance do case studies from ASSISTments provide for implementing adaptive personalization in practice?

Key findings

• Contextual MABs outperform non-contextual ones only when the optimal action truly depends on student features (personalized optimal action model).
• Including extraneous features generally degrades performance and increases exploration costs, especially with many contextual variables.
• In Baseline and Universal optimal action scenarios, contextual personalization can underperform non-contextual approaches, particularly at early horizons.
• When minority group sizes are small, contextual personalization can disproportionately harm minority groups due to higher uncertainty about rare feature values.
• Even with limited features, personalization can substantially improve the proportion of optimal actions for the minority group in the Personalized optimal action model at larger horizons.
• Case studies using ASSISTments data show potential real-world benefits of personalization, emphasizing context and data-driven decisions for feature inclusion.

Figure 2: Average reward per student across 1–10 contextual variables for the two bandit types in the baseline model. In this model, the maximum possible expected reward is $0.6$ , and the expected reward for uniform random assignment is $0.5$ . Error bars represent 1 standard error.