[Paper Review] Bandit Multiclass Linear Classification: Efficient Algorithms for the Separable Case

Alina Beygelzimer, Dávid Pál+4 more|arXiv (Cornell University)|Feb 6, 2019

Advanced Bandit Algorithms ResearchDecision Sciences31 references3 citations

TL;DR

This paper presents efficient algorithms for online multiclass linear classification under bandit feedback, where only binary feedback on prediction correctness is available. It introduces two novel algorithms: one achieving a near-optimal mistake bound of $O(K/ u^2)$ under strong linear separability, and another with a dimension-free mistake bound of $\widetilde{O}(2^{\sqrt{1/\gamma}\log K})$ under weak separability, using kernelized Perceptron and feature mapping to transform weakly separable data into strongly separable form in a higher-dimensional space.

ABSTRACT

We study the problem of efficient online multiclass linear classification with bandit feedback, where all examples belong to one of $K$ classes and lie in the $d$-dimensional Euclidean space. Previous works have left open the challenge of designing efficient algorithms with finite mistake bounds when the data is linearly separable by a margin $γ$. In this work, we take a first step towards this problem. We consider two notions of linear separability: strong and weak. 1. Under the strong linear separability condition, we design an efficient algorithm that achieves a near-optimal mistake bound of $O\left( K/γ^2 ight)$. 2. Under the more challenging weak linear separability condition, we design an efficient algorithm with a mistake bound of $\min (2^{\widetilde{O}(K \log^2 (1/γ))}, 2^{\widetilde{O}(\sqrt{1/γ} \log K)})$. Our algorithm is based on kernel Perceptron, which is inspired by the work of (Klivans and Servedio, 2008) on improperly learning intersection of halfspaces.

Motivation & Objective

• To address the open problem of designing efficient algorithms with finite mistake bounds in online multiclass linear classification under bandit feedback.
• To resolve the challenge of weak linear separability, where prior work lacked efficient algorithms with finite mistake bounds.
• To eliminate explicit dependence on feature dimensionality in mistake bounds, answering an open question posed by Kakade et al. (2008).
• To achieve computationally efficient algorithms that are polynomial in $K$, $d$, $1/\gamma$, and $T$, while maintaining strong generalization guarantees.

Proposed method

• Propose a new algorithm (Algorithm 1) based on running $K$ copies of the binary Perceptron, one per class, to handle strong linear separability.
• Use a kernelized Perceptron approach to transform weakly linearly separable data into strongly separable data in a higher-dimensional feature space via an explicit feature mapping $\phi$.
• Design a positive definite kernel $k(x,x')$ with an $O(d)$ computation time, enabling efficient kernel computation without explicit feature space projection.
• Leverage the kernelized algorithm to achieve a mistake bound dependent on the margin in the transformed space, which is derived from the original weak margin $\gamma$.
• Apply a truncated geometric distribution analysis to bound the expected number of mistakes in adversarial settings with delayed feedback.
• Use the tower property of conditional expectation and probabilistic lower bounds to derive a worst-case lower bound on mistake complexity.

Experimental results

Research questions

• RQ1 Can an efficient algorithm be designed with a finite mistake bound under strong linear separability in the bandit feedback setting?
• RQ2 Is it possible to achieve a finite mistake bound that is independent of the input dimension $d$ under weak linear separability?
• RQ3 Can a computationally efficient algorithm be constructed that maintains a near-optimal mistake bound in the weakly separable case?
• RQ4 What is the optimal trade-off between mistake bound and computational efficiency in the bandit multiclass classification setting?
• RQ5 How can kernel methods be used to transform weakly separable data into strongly separable data to enable efficient learning?

Key findings

• Under strong linear separability, the proposed algorithm achieves a mistake bound of $O(K/\gamma^2)$, which is near-optimal and matches the information-theoretic lower bound up to a constant factor.
• For weak linear separability, the algorithm achieves a mistake bound of $\min(2^{\widetilde{O}(K\log^2(1/\gamma))}, 2^{\widetilde{O}(\sqrt{1/\gamma}\log K)})$, which is dimension-free and improves upon prior work.
• The algorithm is computationally efficient, with both per-round time and memory complexity of $O(dK)$, making it scalable to large $K$ and $d$.
• The kernel mapping transforms weak separability into strong separability in a higher-dimensional space, enabling the use of efficient Perceptron-based learning.
• A lower bound of $\Omega(K(R/\gamma)^2)$ is proven for any algorithm in the strong separability case, showing that the $O(K)$ factor in the mistake bound is unavoidable under bandit feedback.
• The analysis confirms that the extra $O(K)$ factor in the mistake bound compared to the full-information setting is necessary due to the lack of full-label feedback.