[Paper Review] Quantum Speedup in Adaptive Boosting of Binary Classification

Ximing Wang, Yuechi Ma+2 more|arXiv (Cornell University)|Feb 3, 2019

Quantum Computing Algorithms and Architecture5 citations

TL;DR

This paper proposes a quantum algorithm that achieves a quadratic speedup in query complexity for adaptive boosting (AdaBoost) in binary classification by leveraging quantum phase estimation and amplitude amplification. It generalizes AdaBoost to probabilistic weak classifiers and demonstrates that quantum speedup is achievable even when weak learners output probabilistic predictions, with query complexity scaling as O(√N T²) compared to O(N T) classically, where N is the number of training samples and T is the number of iterations.

ABSTRACT

In classical machine learning, a set of weak classifiers can be adaptively combined to form a strong classifier for improving the overall performance, a technique called adaptive boosting (or AdaBoost). However, constructing the strong classifier for a large data set is typically resource consuming. Here we propose a quantum extension of AdaBoost, demonstrating a quantum algorithm that can output the optimal strong classifier with a quadratic speedup in the number of queries of the weak classifiers. Our results also include a generalization of the standard AdaBoost to the cases where the output of each classifier may be probabilistic even for the same input. We prove that the update rules and the query complexity of the non-deterministic classifiers are the same as those of deterministic classifiers, which may be of independent interest to the classical machine-learning community. Furthermore, the AdaBoost algorithm can also be applied to data encoded in the form of quantum states; we show how the training set can be simplified by using the tools of t-design. Our approach describes a model of quantum machine learning where quantum speedup is achieved in finding the optimal classifier, which can then be applied for classical machine-learning applications.

Motivation & Objective

• To develop a quantum algorithm that achieves a quadratic speedup in query complexity for classical AdaBoost in binary classification.
• To generalize AdaBoost to cases where weak classifiers produce probabilistic outputs for the same input.
• To demonstrate that the update rules and query complexity remain unchanged for probabilistic classifiers, preserving classical algorithmic structure.
• To show that quantum state-encoding of training data enables simplification via t-designs, reducing training complexity.
• To establish a framework for quantum machine learning where quantum speedup is achieved in finding optimal classifiers for classical applications.

Proposed method

• The algorithm uses amplitude amplification and phase estimation to estimate the error rate of weak classifiers in superposition, enabling faster convergence.
• It constructs a unitary operation G_t U_⊥ G_t† Z that evolves the state to |cos((2k+1)θ_t)|φ_0⟩|0⟩ + |sin((2k+1)θ_t)|φ_1⟩|1⟩, enabling phase estimation of θ_t.
• The query complexity per iteration is O(ĉ/ε * t), where ĉ is a normalization factor and ε is the target precision.
• The algorithm performs T iterations, each requiring reinitialization after measurement, leading to total query complexity O(ĉ/ε * T²).
• The use of t-designs allows simplification of quantum state-encoding of training data, reducing overhead in state preparation.
• The method ensures that probabilistic weak classifiers do not alter the update rules or query complexity, maintaining consistency with classical AdaBoost.

Experimental results

Research questions

• RQ1 Can quantum algorithms achieve a quadratic speedup in query complexity for adaptive boosting in binary classification?
• RQ2 Does the generalization of AdaBoost to probabilistic weak classifiers preserve the same query complexity and update rules as in the deterministic case?
• RQ3 Can quantum state-encoding of training data be simplified using t-designs to reduce resource overhead?
• RQ4 What is the relationship between the precision of error rate estimation and the number of queries required in the quantum AdaBoost algorithm?
• RQ5 Is the quantum AdaBoost algorithm scalable and effective when applied to real-world machine learning problems with large datasets?

Key findings

• The quantum AdaBoost algorithm achieves a quadratic speedup in query complexity, scaling as O(√N T²) compared to O(N T) for classical AdaBoost, where N is the number of training samples and T is the number of iterations.
• The query complexity for estimating each weak classifier's error rate is O(ĉ/ε), where ĉ = ∏_{t=1}^T max{1/(2˜R_t), 1/(2(1−˜R_t))} and ˜R_t is the error rate at iteration t.
• The algorithm maintains the same update rules and query complexity for probabilistic weak classifiers as for deterministic ones, a result of independent interest to classical machine learning.
• Phase estimation is used to estimate θ_t such that |θ_t − ˆθ_t| ≤ δ, with δ = O(ε/ĉ), ensuring the error in the estimated error rate ˜R_t is bounded by ε.
• The total query complexity is O(ĉ/ε * T²), which is quadratically faster than the classical O(ĉ²/ε² * T) when ĉ is large and T is small.
• The use of t-designs enables efficient encoding of training data into quantum states, simplifying the state preparation process and reducing overhead.