[Paper Review] Hierarchical Contextual Uplift Bandits for Catalog Personalization

A. Agrawal, Rajesh Kumar Mohanty+2 more|arXiv (Cornell University)|Jan 20, 2026

Advanced Bandit Algorithms Research0 citations

TL;DR

The paper introduces a Hierarchical Contextual Uplift Bandit (HCUB) framework that uses a coarse-to-fine context hierarchy and uplift-based rewards to personalize Dream11's catalog, achieving revenue uplift in online A/B tests and improving adaptation in dynamic environments.

ABSTRACT

Contextual Bandit (CB) algorithms are widely adopted for personalized recommendations but often struggle in dynamic environments typical of fantasy sports, where rapid changes in user behavior and dramatic shifts in reward distributions due to external influences necessitate frequent retraining. To address these challenges, we propose a Hierarchical Contextual Uplift Bandit framework. Our framework dynamically adjusts contextual granularity from broad, system-wide insights to detailed, user-specific contexts, using contextual similarity to facilitate effective policy transfer and mitigate cold-start issues. Additionally, we integrate uplift modeling principles into our approach. Results from large-scale A/B testing on the Dream11 fantasy sports platform show that our method significantly enhances recommendation quality, achieving a 0.4% revenue improvement while also improving user satisfaction metrics compared to the current production system. We subsequently deployed this system to production as the default catalog personalization system in May 2025 and observed a further 0.5% revenue improvement.

Motivation & Objective

• Address rapid environmental changes and cold-start issues in contextual bandits for dynamic fantasy sports catalogs.
• Provide a hierarchical context representation to enable policy transfer across system- and user-level contexts.
• Incorporate uplift modeling into the objective to optimize incremental business impact (revenue, engagement, retention).
• Evaluate Online and Offline performance, including production deployment, and quantify the impact on revenue and engagement.

Proposed method

• Propose a hierarchical context tree that spans system-level to user-level features for context representation.
• Define a reward function as a weighted sum of uplifts in short-term engagement, retention proxy, and revenue.
• Use reward inheritance: propagate uplift signals top-down through the hierarchy to mitigate cold-start and stabilize exploration.
• Estimate uplifts via hierarchical bootstrapping to obtain medians and confidence intervals for each node-action pair.
• Perform Bayesian UCB at leaf nodes to select catalog actions, balancing exploration and exploitation.
• Evaluate the approach with online A/B tests and offline simulations, including ablations removing reward inheritance.

Experimental results

Research questions

• RQ1 Does HCUB with hierarchical context and uplift rewards improve revenue and engagement over the production catalog personalization system?
• RQ2 What is the impact of hierarchical reward inheritance on learning speed and adaptation in non-stationary, dynamic environments?
• RQ3 How does HCUB perform in large-scale online deployment versus offline simulations in terms of regret and business metrics?
• RQ4 Can reward inheritance stabilize learning across user cohorts and system-level contexts to mitigate cold-start?

Key findings

• Online experiments show revenue uplift with HCUB: +0.42% in Feb–Mar 2025 (6M users) and +0.51% May–Present (entire Dream11 base).
• DAU changes were small and statistically insignificant: +0.05% (Feb–Mar) and +0.1% (May–Present).
• Offline simulations indicate 4–5% regret improvement when using reward inheritance versus not using inheritance.
• Production deployment achieved revenue improvement, validating production viability of HCUB.
• The approach demonstrates significant revenue gains while maintaining or improving short- and long-term engagement metrics.