[Paper Review] Robust Stochastic Linear Contextual Bandits Under Adversarial Attacks

Qin Ding, Cho‐Jui Hsieh+1 more|arXiv (Cornell University)|Jun 5, 2021

Advanced Bandit Algorithms Research20 references4 citations

TL;DR

This paper proposes the first robust stochastic linear contextual bandit algorithm that achieves sub-linear regret under fully adaptive, omniscient adversarial attacks on both rewards and context, without requiring prior knowledge of the attack budget or attack form. The method uses a two-layer bandit structure with dynamic exploration adjustment, significantly improving robustness over LinUCB, LinTS, and Greedy baselines in simulations and real-world datasets like Movielens and Netflix under Garcelon and Oracle attacks.

ABSTRACT

Stochastic linear contextual bandit algorithms have substantial applications in practice, such as recommender systems, online advertising, clinical trials, etc. Recent works show that optimal bandit algorithms are vulnerable to adversarial attacks and can fail completely in the presence of attacks. Existing robust bandit algorithms only work for the non-contextual setting under the attack of rewards and cannot improve the robustness in the general and popular contextual bandit environment. In addition, none of the existing methods can defend against attacked context. In this work, we provide the first robust bandit algorithm for stochastic linear contextual bandit setting under a fully adaptive and omniscient attack with sub-linear regret. Our algorithm not only works under the attack of rewards, but also under attacked context. Moreover, it does not need any information about the attack budget or the particular form of the attack. We provide theoretical guarantees for our proposed algorithm and show by experiments that our proposed algorithm improves the robustness against various kinds of popular attacks.

Motivation & Objective

• To address the lack of robustness in stochastic linear contextual bandit algorithms under adversarial attacks, particularly when both rewards and context are manipulated.
• To develop a method that maintains sub-linear regret under fully adaptive and omniscient attackers who observe the player's decisions in real time.
• To design an algorithm that does not require knowledge of the attack budget or attack pattern, making it applicable to general contextual bandit settings.
• To extend robustness beyond reward corruption to include context manipulation, a previously unaddressed challenge in contextual bandits.
• To empirically validate the algorithm's superiority over LinUCB, LinTS, and Greedy methods under diverse attack types on real-world recommendation data.

Proposed method

• Proposes a two-layer bandit architecture: an upper EXP3 layer dynamically selects an exploration parameter C' for the lower LinUCB layer.
• The EXP3 layer chooses C' every round based on immediate feedback from the previous round, avoiding epoch-based restarting to preserve historical information.
• Introduces a modified LinUCB algorithm with an enlarged exploration parameter that depends on C', enabling robustness to adversarial attacks.
• Theoretical analysis shows a regret bound of Õ(d^{1/2}(C+1)T^{3/4}) + Õ(d^{3/2}CT^{1/4}), which is sub-linear under finite attack budget C.
• Empirically evaluates the method via BOB-No-Restart, a variant without restarting that performs better in practice despite lacking theoretical guarantees.
• Uses real-world datasets (Movielens, Netflix) with feature vectors derived from matrix factorization to simulate realistic contextual bandit environments.

Experimental results

Research questions

• RQ1 Can a stochastic linear contextual bandit algorithm achieve sub-linear regret under fully adaptive, omniscient attacks on both rewards and context?
• RQ2 Is it possible to design a robust algorithm without requiring prior knowledge of the attack budget or attack pattern?
• RQ3 How does the performance of the proposed algorithm compare to LinUCB, LinTS, and Greedy under reward and context attacks?
• RQ4 Can a restarting-free variant of the algorithm (BOB-No-Restart) achieve superior empirical robustness despite lacking theoretical guarantees?
• RQ5 What is the theoretical trade-off between attack budget C and time horizon T in the regret bound for robust contextual bandits?

Key findings

• The proposed RobustBandit algorithm achieves sub-linear regret Õ(d^{1/2}(C+1)T^{3/4}) + Õ(d^{3/2}CT^{1/4}) under adversarial attacks on both rewards and context.
• The algorithm outperforms LinUCB, LinTS, and Greedy baselines in terms of cumulative regret across simulations, Movielens, and Netflix datasets under Garcelon and Oracle attacks.
• BOB-No-Restart, a restarting-free variant, empirically achieves significantly better robustness than all baselines, including the theoretically guaranteed RobustBandit.
• Traditional LinUCB suffers from linear regret when the attack budget C is Ω(log T), demonstrating the fragility of standard algorithms under attack.
• The method is the first to defend against both reward and context attacks in the general contextual bandit setting without requiring attack budget or form information.
• Experiments confirm that the algorithm maintains robust performance even when attackers target the top half of arms, indicating strong resilience to strategic, adaptive attacks.