[Paper Review] (Locally) Differentially Private Combinatorial Semi-Bandits

Xiaoyu Chen, Kai Zheng+4 more|arXiv (Cornell University)|Jun 1, 2020

Advanced Bandit Algorithms Research37 references4 citations

TL;DR

This paper proposes differentially private algorithms for Combinatorial Semi-Bandits under $B_\infty$- and $B_1$-bounded smoothness assumptions, achieving near-optimal regret bounds that match non-private rates. It demonstrates that privacy does not incur an additional cost in regret for these settings, with optimal bounds of $\Theta(\frac{mB^{2}_{\infty}\ln T}{\varepsilon^{2}\Delta})$ under $\varepsilon$-LDP and $\tilde{\Theta}(\frac{mB^{2}_{\infty}\ln T}{\varepsilon\Delta})$ under $\varepsilon$-DP, and $\tilde{\Theta}(\frac{mKB^{2}_{1}\ln T}{\varepsilon\Delta})$ for $B_1$-bounded smoothness under $\varepsilon$-DP.

ABSTRACT

In this paper, we study Combinatorial Semi-Bandits (CSB) that is an extension of classic Multi-Armed Bandits (MAB) under Differential Privacy (DP) and stronger Local Differential Privacy (LDP) setting. Since the server receives more information from users in CSB, it usually causes additional dependence on the dimension of data, which is a notorious side-effect for privacy preserving learning. However for CSB under two common smoothness assumptions \cite{kveton2015tight,chen2016combinatorial}, we show it is possible to remove this side-effect. In detail, for $B_{\infty}$-bounded smooth CSB under either $\varepsilon$-LDP or $\varepsilon$-DP, we prove the optimal regret bound is $Θ(\frac{mB^2_{\infty}\ln T } {Δε^2})$ or $ ildeΘ(\frac{mB^2_{\infty}\ln T} { Δε})$ respectively, where $T$ is time period, $Δ$ is the gap of rewards and $m$ is the number of base arms, by proposing novel algorithms and matching lower bounds. For $B_1$-bounded smooth CSB under $\varepsilon$-DP, we also prove the optimal regret bound is $ ildeΘ(\frac{mKB^2_1\ln T} {Δε})$ with both upper bound and lower bound, where $K$ is the maximum number of feedback in each round. All above results nearly match corresponding non-private optimal rates, which imply there is no additional price for (locally) differentially private CSB in above common settings.

Motivation & Objective

• To design differentially private algorithms for Combinatorial Semi-Bandits (CSB) under local and global differential privacy (LDP and DP) settings.
• To analyze the regret bounds of private CSB under common smoothness assumptions: $B_\infty$-bounded and $B_1$-bounded smoothness.
• To determine whether privacy imposes an additional regret cost beyond the non-private optimal rates in CSB.
• To establish nearly matching upper and lower bounds for regret under various privacy and smoothness conditions.

Proposed method

• Proposes novel private algorithms for $B_\infty$-bounded smooth CSB under $\varepsilon$-LDP and $\varepsilon$-DP, using private summation and tree-based aggregation techniques.
• Introduces a new algorithm for $B_1$-bounded smooth CSB under $\varepsilon$-DP, achieving a regret upper bound of $\mathcal{O}(\frac{mK^{2}B^{2}_{1}\ln T}{\Delta\varepsilon^{2}})$.
• Employs a coupling argument based on Karwa & Vadhan (2017) to derive lower bounds on the number of pulls of sub-optimal super arms.
• Constructs a hard instance with $m$ base arms divided into optimal, public, and auxiliary sets to prove $\Omega(\frac{mKB_{1}\ln T}{\varepsilon\Delta})$ regret lower bound for $B_1$-bounded CSB under $\varepsilon$-DP.
• Uses sparse vector technique and successively eliminated arms to reduce regret in private CSB, inspired by non-private Successive Elimination.
• Applies a novel reduction from private CSB to private MAB with structured feedback, leveraging smoothness to avoid dimension-dependent regret blowup.

Experimental results

Research questions

• RQ1 Can differentially private algorithms for Combinatorial Semi-Bandits achieve regret bounds that match the non-private optimal rates?
• RQ2 Does the dimensionality of feedback (i.e., $K$) introduce an unavoidable regret penalty in private CSB under $B_\infty$-bounded smoothness?
• RQ3 What is the optimal regret bound for $B_1$-bounded smooth CSB under $\varepsilon$-DP, and can it be achieved without additional dependence on $K$?
• RQ4 Is the regret lower bound for $B_1$-bounded smooth CSB under $\varepsilon$-DP tight, or is there room for improvement in the current lower bound?

Key findings

• For $B_\infty$-bounded smooth CSB, the optimal regret under $\varepsilon$-LDP is $\Theta(\frac{mB^{2}_{\infty}\ln T}{\varepsilon^{2}\Delta})$, matching the non-private rate up to logarithmic factors.
• Under $\varepsilon$-DP, the optimal regret for $B_\infty$-bounded smooth CSB is $\tilde{\Theta}(\frac{mB^{2}_{\infty}\ln T}{\varepsilon\Delta})$, nearly matching the non-private bound.
• For $B_1$-bounded smooth CSB under $\varepsilon$-DP, the optimal regret is $\tilde{\Theta}(\frac{mKB^{2}_{1}\ln T}{\varepsilon\Delta})$, with both upper and lower bounds matching up to logarithmic factors.
• The regret lower bound for $B_1$-bounded smooth CSB under $\varepsilon$-DP is $\Omega(\frac{mKB_{1}\ln T}{\varepsilon\Delta})$, which is tight up to logarithmic factors.
• The proposed algorithm for $B_1$-bounded smooth CSB under $\varepsilon$-DP achieves $\mathcal{O}(\frac{mK^{2}B^{2}_{1}\ln T}{\Delta\varepsilon^{2}})$ regret, suggesting a gap between upper and lower bounds in the $K$-dependence.
• The paper conjectures that the lower bound is loose and that the algorithm is near-optimal, with the gap in $K$-dependence remaining an open problem.