[Paper Review] Differentially-Private Federated Linear Bandits

Abhimanyu Dubey, Alex Pentland|arXiv (Cornell University)|Oct 22, 2020

Advanced Bandit Algorithms Research51 references32 citations

TL;DR

Introduces FedUCB, a differentially-private federated contextual linear bandit algorithm for centralized and decentralized settings, with regret and privacy guarantees and analysis of communication complexity.

ABSTRACT

The rapid proliferation of decentralized learning systems mandates the need for differentially-private cooperative learning. In this paper, we study this in context of the contextual linear bandit: we consider a collection of agents cooperating to solve a common contextual bandit, while ensuring that their communication remains private. For this problem, we devise extsc{FedUCB}, a multiagent private algorithm for both centralized and decentralized (peer-to-peer) federated learning. We provide a rigorous technical analysis of its utility in terms of regret, improving several results in cooperative bandit learning, and provide rigorous privacy guarantees as well. Our algorithms provide competitive performance both in terms of pseudoregret bounds and empirical benchmark performance in various multi-agent settings.

Motivation & Objective

• Motivate practical private cooperative learning in decentralized (federated) contextual bandits.
• Formally define federated differential privacy for multi-agent contextual bandits.
• Propose FedUCB variants for centralized and decentralized settings with privacy guarantees.
• Analyze regret bounds and communication complexity under privacy constraints.
• Provide guidance on how privacy budgets affect performance in multi-agent networks.

Proposed method

• Extend LinUCB to a federated setting by constructing time-varying ellipsoidal confidence sets and UCBs.
• Perturb Gram matrices and reward vectors with privacy-preserving noise to satisfy differential privacy.
• Develop centralized FedUCB with a controller that synchronizes privatized global statistics and controls heterogeneity.
• Adapt FedUCB to a decentralized peer-to-peer network with delayed, local synchronization and a gamma-delay protocol.
• Introduce a Privatizer subroutine that implements a tree-based mechanism for private release of cumulative statistics.
• Derive regret bounds that relate privacy parameters (ε, δ) to exploration, communication, and network topology.

Experimental results

Research questions

• RQ1 How can one design differentially-private federated algorithms for contextual linear bandits that work in both centralized and decentralized settings?
• RQ2 What are the regret guarantees (group pseudoregret) when privacy constraints are imposed on multi-agent bandit learning?
• RQ3 How does communication relate to privacy budget and regret in federated contextual bandits?
• RQ4 What is the impact of network topology and synchronization on privacy-preserving federated bandit performance?
• RQ5 Can the proposed FedUCB framework achieve near-optimal rates comparable to non-private multi-agent contextual bandits under privacy constraints?

Key findings

• Centralized FedUCB achieves a high-probability group pseudoregret bound that matches non-private bounds in its T and M dependence up to polylog factors.
• Decentralized FedUCB introduces a delay-aware, topology-dependent bound that includes a diameter-based factor of the communication graph.
• The framework provides explicit privacy guarantees for federated contexts and rewards through a Privatizer subroutine based on tree-based differential privacy mechanisms.
• A trade-off is established between communication rounds and regret: with fewer synchronizations, regret grows, while more frequent synchronization can improve utility.
• The analysis connects privacy budgets to spectral properties of perturbations, showing how ε, δ affect regret via ρ_min, ρ_max, and κ in the perturbation design.
• Communication complexity bounds relate the number of synchronizations to dimension, horizon, and privacy parameters, providing guidance for practical deployment.