[Paper Review] Truly Adapting to Adversarial Constraints in Constrained MABs

Francesco Emanuele Stradi, Kalana Kalupahana+3 more|arXiv (Cornell University)|Feb 16, 2026

Advanced Bandit Algorithms Research0 citations

TL;DR

The paper studies constrained multi-armed bandits with unknown, potentially adversarial constraints and non-stationary losses. It provides algorithms achieving sublinear regret and constraint violation that degrade only with the constraint non-stationarity, with full and bandit feedback settings.

ABSTRACT

We study the constrained variant of the \emph{multi-armed bandit} (MAB) problem, in which the learner aims not only at minimizing the total loss incurred during the learning dynamic, but also at controlling the violation of multiple \emph{unknown} constraints, under both \emph{full} and \emph{bandit feedback}. We consider a non-stationary environment that subsumes both stochastic and adversarial models and where, at each round, both losses and constraints are drawn from distributions that may change arbitrarily over time. In such a setting, it is provably not possible to guarantee both sublinear regret and sublinear violation. Accordingly, prior work has mainly focused either on settings with stochastic constraints or on relaxing the benchmark with fully adversarial constraints (\emph{e.g.}, via competitive ratios with respect to the optimum). We provide the first algorithms that achieve optimal rates of regret and \emph{positive} constraint violation when the constraints are stochastic while the losses may vary arbitrarily, and that simultaneously yield guarantees that degrade smoothly with the degree of adversariality of the constraints. Specifically, under \emph{full feedback} we propose an algorithm attaining $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ regret and $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ {positive} violation, where $C$ quantifies the amount of non-stationarity in the constraints. We then show how to extend these guarantees when only bandit feedback is available for the losses. Finally, when \emph{bandit feedback} is available for the constraints, we design an algorithm achieving $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ {positive} violation and $\widetilde{\mathcal{O}}(\sqrt{T}+C\sqrt{T})$ regret.

Motivation & Objective

• Understand the impact of unknown, time-varying constraint distributions on constrained MABs.
• Develop algorithms that achieve sublinear regret and sublinear positive constraint violations when losses may be adversarial but constraints are stochastic.
• Provide extensions to bandit feedback for losses and constraints.
• Characterize how violation and regret bounds degrade with the non-stationarity level C of the constraints.

Proposed method

• Introduce a corruption level C to quantify non-stationarity in constraints.
• Construct per-round approximate feasible sets X_t using optimistic estimates of constraint violations.
• Use online mirror descent with fixed-share updates to handle moving decision spaces and achieve switching regret guarantees.
• Develop a two-phase approach for bandit feedback on losses to ensure sufficient exploration (ExpOpt-ConOMD).
• Extend to bandit feedback on constraints by adapting confidence bounds and exploration strategies (Constrained OMD variants).
• Provide theoretical bounds showing R_T = Ŝ(√T + C) and V_T = Ŝ(√T + C) under full feedback, with extensions to bandit settings yielding analogous or slightly weaker guarantees.

Experimental results

Research questions

• RQ1 Can sublinear regret and sublinear positive constraint violation be achieved when constraints are unknown and non-stationary, with losses possibly adversarial?
• RQ2 How should the learner adaptively construct feasible action sets to cope with unknown constraint corruption while maintaining tractable regret?
• RQ3 What are the optimal regret and violation bounds under full feedback versus bandit feedback for losses and for constraints?
• RQ4 How do bounds degrade with the degree of constraint non-stationarity (C) and can they be made to degrade smoothly?

Key findings

• Under full feedback, the proposed algorithm ConOMD-FS achieves regret and positive constraint violation of order Ŝ(√T + C).
• When only bandit feedback is available for losses, the ConOMD-FS approach extends to obtain similar guarantees with adjusted analysis.
• With bandit feedback on constraints, the ExpOpt-ConOMD family achieves Ŝ(√T + C) positive violation and Ŝ(√T + C√T) regret (choosing β = 1/2 yields this bound).
• The corruption level C is shown to be the primary driver of degradation in performance, and guarantees degrade smoothly with C rather than catastrophically.
• The proposed methods handle moving (time-varying) constraint futures via switching regret on moving decision spaces and a doubling trick over phases.
• Compared to prior work, the results match best-of-both-worlds guarantees in stochastic settings and provide sublinear regret under mildly adversarial constraints.