[Paper Review] Quantum-Assisted Support Vector Regression

Archismita Dalal, Mohsen Bagherimehrab+1 more|arXiv (Cornell University)|Nov 17, 2021

Quantum Information and Cryptography4 citations

TL;DR

This paper proposes a quantum-assisted support vector regression (SVR) model using D-Wave's quantum-classical hybrid solver to improve facial landmark detection accuracy on small datasets. By formulating the SVR training problem as a quadratic unconstrained binary optimization (QUBO) problem and solving it via quantum annealing, the method achieves slightly higher detection accuracy and significantly lower variance than classical SVR and state-of-the-art baselines, demonstrating a proof-of-concept for quantum-enhanced regression in supervised learning.

ABSTRACT

A popular machine-learning model for regression tasks, including stock-market prediction, weather forecasting and real-estate pricing, is the classical support vector regression (SVR). However, a practically realisable quantum SVR remains to be formulated. We devise annealing-based algorithms, namely simulated and quantum-classical hybrid, for training two SVR models and compare their empirical performances against the SVR implementation of Python's scikit-learn package for facial-landmark detection (FLD), a particular use case for SVR. Our method is to derive a quadratic-unconstrained-binary formulation for the optimisation problem used for training a SVR model and solve this problem using annealing. Using D-Wave's hybrid solver, we construct a quantum-assisted SVR model, thereby demonstrating a slight advantage over classical models regarding FLD accuracy. Furthermore, we observe that annealing-based SVR models predict landmarks with lower variances compared to the SVR models trained by gradient-based methods. Our work is a proof-of-concept example for applying quantum-assisted SVR to a supervised-learning task with a small training dataset.

Motivation & Objective

• To develop a quantum-assisted SVR model using quantum annealing for supervised learning tasks with limited training data.
• To evaluate the performance of quantum-annealing-based SVR against classical SVR and state-of-the-art FLD algorithms on facial landmark detection.
• To investigate whether quantum annealing can yield lower prediction variance and higher accuracy than classical greedy optimization in regression models.
• To demonstrate a QUBO formulation of the SVR training problem suitable for implementation on current quantum annealing hardware.
• To explore the feasibility of quantum-assisted machine learning for real-world computer vision tasks with small datasets.

Proposed method

• The SVR training problem is reformulated as a quadratic unconstrained binary optimization (QUBO) problem to enable solution via quantum annealing.
• The multi-output facial landmark detection task is decomposed into multiple single-output regression problems, each trained with a separate SVR model.
• Both classical simulated annealing and D-Wave's quantum-classical hybrid annealing solver are used to solve the QUBO formulation.
• The final landmark prediction is obtained by averaging predictions from 20 feasible solutions generated via annealing, reducing variance.
• The model is trained and tested on a small dataset of 100 unconstrained face images, with performance evaluated on LFPW and BioID benchmark datasets.
• The QUBO formulation is derived from the Lagrangian dual of the ε-SVR optimization problem, enabling mapping to Ising spin glass problems solvable by D-Wave's annealer.

Figure 1: Two slack variables $\xi^{+}$ and $\xi^{-}$ are used in an $\varepsilon$ -SVR formulation. Red circles are data points. (left) $\xi^{+}=0$ if the corresponding training data point is below the upper bound and $\xi^{+}>0$ if it is above the upper bound, (right) $\xi^{-}=0$ if the correspond

Experimental results

Research questions

• RQ1 Can quantum annealing improve the accuracy of support vector regression in small-data supervised learning tasks?
• RQ2 Does quantum-assisted SVR reduce prediction variance compared to classical greedy optimization in regression models?
• RQ3 How does the performance of a quantum-annealing-based SVR model compare to classical SVR and state-of-the-art FLD algorithms on facial landmark detection?
• RQ4 Can a QUBO formulation of the SVR training problem be effectively solved using current quantum-classical hybrid solvers?
• RQ5 What is the impact of using an ensemble of near-optimal solutions from quantum annealing on model generalization and robustness?

Key findings

• The quantum-assisted SVR model achieved a slight but measurable improvement in landmark detection accuracy compared to classical SVR and the BoRMaN baseline on the LFPW and BioID datasets.
• Annealing-based SVR models produced predictions with significantly lower variance than those from greedy-optimized classical models, due to global optimization of the Lagrangian.
• The hybrid annealing approach generated an average of 20 feasible solutions, which contributed to reduced mean normalized deviation error (MNDE) variance in test predictions.
• The QUBO formulation of the SVR training problem was successfully implemented and solved using D-Wave’s Hybrid Solver, validating its feasibility on current quantum hardware.
• The method demonstrated that quantum-assisted optimization can enhance regression performance even with limited training data, offering a proof-of-concept for quantum machine learning in supervised tasks.
• Future improvements may include optimizing annealing hyperparameters, increasing image segmentation granularity, and exploring fully quantum annealing workflows on larger devices.

Figure 2: An example image from the LFW database [ 37 ] . Using data from Ref. [ 56 ] , we show the five landmarks with manually determined positions, labels 1–5, and the face box as a red rectangle.