[Paper Review] Graph Clustering Bandits for Recommendation

Shuai Li, Claudio Gentile+1 more|arXiv (Cornell University)|May 2, 2016

Advanced Bandit Algorithms Research12 references11 citations

TL;DR

This paper proposes GCLUB, a graph clustering bandit algorithm that dynamically groups users based on behavioral similarity in multi-armed bandit settings, enabling joint exploration over both items and user clusters. It significantly improves recommendation performance—especially for low-activity users—by combining contextual bandits with adaptive clustering, outperforming state-of-the-art methods on four real-world datasets including LastFM, Delicious, MovieLens, and Yahoo News.

ABSTRACT

We investigate an efficient context-dependent clustering technique for recommender systems based on exploration-exploitation strategies through multi-armed bandits over multiple users. Our algorithm dynamically groups users based on their observed behavioral similarity during a sequence of logged activities. In doing so, the algorithm reacts to the currently served user by shaping clusters around him/her but, at the same time, it explores the generation of clusters over users which are not currently engaged. We motivate the effectiveness of this clustering policy, and provide an extensive empirical analysis on real-world datasets, showing scalability and improved prediction performance over state-of-the-art methods for sequential clustering of users in multi-armed bandit scenarios.

Motivation & Objective

• To address the challenge of sparse user activity in recommendation systems, where most users have little or no historical interaction data.
• To improve recommendation performance by incorporating collaborative effects through dynamic clustering of users based on behavioral similarity.
• To extend standard contextual bandits by adding exploration not only over items but also over user cluster assignments, particularly for inactive or new users.
• To evaluate the effectiveness of joint exploration over items and clusters in unbalanced user activity regimes.

Proposed method

• The algorithm models users as nodes in a graph, where edges represent behavioral similarity derived from past item selections.
• It uses a stochastic exploration mechanism to dynamically form and reconfigure user clusters in real time based on observed user behavior.
• The system applies a contextual bandit framework where each user’s item recommendations are informed by cluster-level preference patterns, with exploration over cluster assignments.
• It maintains separate models for each cluster and uses a UCB-style exploration strategy to balance exploration of new clusters and known cluster behaviors.
• The method integrates both user and item features, using a linear model to predict payoff (e.g., click-through rate) for item recommendations within each cluster.
• Parameter tuning is performed via grid search on training data, with performance evaluated on test sets using cumulative regret and click-through rate (CTR) metrics.

Experimental results

Research questions

• RQ1 Can dynamic clustering of users based on behavioral similarity improve recommendation performance in multi-armed bandit settings?
• RQ2 Does adding exploration over user cluster assignments—beyond standard item-level exploration—lead to measurable gains in prediction accuracy?
• RQ3 How does the proposed method perform in real-world settings with highly unbalanced user activity levels?
• RQ4 In what scenarios does GCLUB outperform LinUCB-ONE, LinUCB-IND, and CLUB in terms of cumulative regret and CTR?

Key findings

• GCLUB outperforms all baselines, including CLUB, LinUCB-ONE, and LinUCB-IND, across all four real-world datasets: LastFM, Delicious, MovieLens, and Yahoo News.
• On LastFM, Delicious, and MovieLens, GCLUB achieves a lower ratio of cumulative regret to the random baseline (RAN), indicating superior regret minimization.
• On the Yahoo dataset, where only logged feedback is available, GCLUB achieves a higher click-through rate (CTR) ratio relative to RAN than all competitors, demonstrating improved real-world performance.
• The performance gain is most pronounced in scenarios with high user activity imbalance, where GCLUB effectively handles low-activity and new users through dynamic cluster exploration.
• CLUB, while effective, is outperformed by GCLUB in all datasets, especially in the long-term, indicating that joint exploration over clusters enhances stability and accuracy.
• The results suggest that combining collaborative filtering via clustering with bandit exploration at the cluster level leads to a significant and consistent improvement in recommendation quality.