[Paper Review] Regret Analysis of Sleeping Competing Bandits

Shinnosuke Uba, Yutaro Yamaguchi|arXiv (Cornell University)|Mar 20, 2026

Advanced Bandit Algorithms Research0 citations

TL;DR

The paper defines sleeping competing bandits, derives regret lower bounds and proposes algorithms (AC-UCB and AC-ETGS) that achieve sublinear player-regrets under reasonable assumptions, achieving asymptotic optimality when the number of arms K grows relative to the number of players N.

ABSTRACT

The Competing Bandits framework is a recently emerging area that integrates multi-armed bandits in online learning with stable matching in game theory. While conventional models assume that all players and arms are constantly available, in real-world problems, their availability can vary arbitrarily over time. In this paper, we formulate this setting as Sleeping Competing Bandits. To analyze this problem, we naturally extend the regret definition used in existing competing bandits and derive regret bounds for the proposed model. We propose an algorithm that simultaneously achieves an asymptotic regret bound of $\mathrm{O}\left(NK\log T_{i}/Δ^2 ight)$ under reasonable assumptions, where $N$ is the number of players, $K$ is the number of arms, $T_{i}$ is the number of rounds of each player $p_i$, and $Δ$ is the minimum reward gap. We also provide a regret lower bound of $\mathrmΩ\left( N(K-N+1)\log T_{i}/Δ^2 ight)$ under the same assumptions. This implies that our algorithm is asymptotically optimal in the regime where the number of arms $K$ is relatively larger than the number of players $N$.

Motivation & Objective

• Formulate the sleeping competing bandit setting where both players and arms can be unavailable over time.
• Define player-optimal and player-pessimal stable regrets in this dynamic, two-sided market.
• Establish fundamental regret lower bounds for any algorithm in this setting.
• Develop centralized algorithms extending UCB/ETGS to sleeping environments and analyze their regret.
• Characterize regimes (e.g., K relative to N) where the proposed methods are asymptotically optimal.

Proposed method

• Define the Sleeping Competing Bandits model with time-varying availability for players and arms.
• Use stable matching concepts (GS algorithm) to assign arms to players under capacity constraints.
• Employ Upper and Lower Confidence Bounds (UCB/LCB) to guide arm rankings per player.
• Propose Awake Centralized UCB (AC-UCB) that learns preferences and runs player-proposing GS per round.
• Propose Awake Centralized Explore-Then-Gale–Shapley (AC-ETGS) that alternates exploration and exploitation rounds using ETGS criteria.
• Prove sublinear upper bounds for regret under certain conditions and derive a matching lower bound to show asymptotic optimality in regimes where K is large relative to N.

Figure 1: Regret comparison between random and weighted exploration with heterogeneous player unavailability probabilities.

Experimental results

Research questions

• RQ1 What are the fundamental regret limits for sleeping competing bandits when availability of players and arms varies arbitrarily over time?
• RQ2 Can centralized algorithms be designed to achieve sublinear player-optimal and player-pessimal stable regrets in the sleeping setting?
• RQ3 How do existing regret bounds extend to the sleeping version of competing bandits, and how do they scale with N, K, T, and Δ?
• RQ4 Under what conditions (e.g., K relative to N) do the proposed algorithms become asymptotically optimal?
• RQ5 What are the implications of arm capacity and dynamic preferences on stability and regret?

Key findings

• Any policy cannot achieve strictly sublinear regret without additional assumptions (alpha-consistency failure without assumptions).
• Under reasonable assumptions, the player-pessimal stable regret lower bound is Omega(N(K−N+1) log Ti / Δ^2).
• The AC-UCB algorithm achieves an upper bound of O(NK log Ti / Δ^2) for player-pessimal stable regret.
• The AC-ETGS algorithm achieves an upper bound of O(NK^2 log Ti / Δ^2) for player-optimal stable regret.
• The bounds imply asymptotic optimality of the proposed methods in the regime where K is relatively larger than N (K = O(log Ti) in analysis).
• A fundamental lower bound is Omega(N(K−N+1) log Ti / Δ^2) under the same assumptions.

Figure 2: Regret comparison between random and weighted exploration with identical player unavailability probabilities.