[Paper Review] Quantum Machine Learning Architecture Search via Deep Reinforcement Learning

Xin Dai, Tzu-Chieh Wei+2 more|arXiv (Cornell University)|Jul 29, 2024

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper proposes a deep reinforcement learning framework, RL-QMLAS, to automatically search for optimal variational quantum circuit (VQC) architectures for supervised classification tasks. By using an adaptive learning target mechanism, the agent discovers high-accuracy, low-depth quantum circuits without prior physical knowledge, achieving performance comparable to classical models with far fewer parameters.

ABSTRACT

The rapid advancement of quantum computing (QC) and machine learning (ML) has given rise to the burgeoning field of quantum machine learning (QML), aiming to capitalize on the strengths of quantum computing to propel ML forward. Despite its promise, crafting effective QML models necessitates profound expertise to strike a delicate balance between model intricacy and feasibility on Noisy Intermediate-Scale Quantum (NISQ) devices. While complex models offer robust representation capabilities, their extensive circuit depth may impede seamless execution on extant noisy quantum platforms. In this paper, we address this quandary of QML model design by employing deep reinforcement learning to explore proficient QML model architectures tailored for designated supervised learning tasks. Specifically, our methodology involves training an RL agent to devise policies that facilitate the discovery of QML models without predetermined ansatz. Furthermore, we integrate an adaptive mechanism to dynamically adjust the learning objectives, fostering continuous improvement in the agent's learning process. Through extensive numerical simulations, we illustrate the efficacy of our approach within the realm of classification tasks. Our proposed method successfully identifies VQC architectures capable of achieving high classification accuracy while minimizing gate depth. This pioneering approach not only advances the study of AI-driven quantum circuit design but also holds significant promise for enhancing performance in the NISQ era.

Motivation & Objective

• To address the challenge of designing effective quantum machine learning (QML) models on Noisy Intermediate-Scale Quantum (NISQ) devices, which requires expert knowledge and balances model complexity with hardware feasibility.
• To automate the design of variational quantum circuit (VQC) architectures for supervised learning tasks, eliminating reliance on manually crafted ansatzes.
• To improve the efficiency and scalability of quantum architecture search by dynamically adjusting learning objectives during training.
• To demonstrate that RL-based search can yield high-accuracy, shallow-circuit VQCs suitable for current noisy quantum hardware.
• To reduce the parameterization burden of quantum classifiers compared to classical models, enhancing their practicality.

Proposed method

• The framework employs a deep reinforcement learning (RL) agent to explore and generate quantum gate sequences that form VQC architectures for classification.
• The RL agent is trained using a reward signal based on classification accuracy, with an adaptive learning target that dynamically increases the required accuracy threshold to sustain learning progress.
• The architecture search is performed in a discrete action space defined by gate types (single-qubit rotations and entangling CNOT gates) and qubit connectivity.
• The agent's policy network is trained via proximal policy optimization (PPO), a stable deep RL algorithm suitable for continuous and discrete control tasks.
• The search process avoids predefined ansatz structures, enabling discovery of novel, task-specific circuit designs.
• The method integrates a dynamic reward adjustment mechanism that prevents premature convergence and enhances exploration of high-performing architectures.

Experimental results

Research questions

• RQ1 Can deep reinforcement learning effectively discover high-performing VQC architectures for classification without prior physical knowledge?
• RQ2 How does an adaptive learning target mechanism improve the convergence and performance of RL-based quantum architecture search?
• RQ3 To what extent can RL-QMLAS generate shallow, low-parameter VQCs that match or exceed classical ML models in accuracy?
• RQ4 How does the framework generalize across different datasets with varying complexity, such as make_classification and make_moons?
• RQ5 What is the impact of noise and hardware constraints on the deployability of RL-discovered circuits in real NISQ devices?

Key findings

• The RL-QMLAS framework successfully discovered VQC architectures that achieved 90.3% classification accuracy on the 4-dimensional make_classification dataset, matching the performance of logistic regression and SVM models.
• On the 2-dimensional make_moons dataset, the quantum classifier achieved 100% accuracy with only 2 quantum gates, comparable to the SVM's performance using 44 support vectors.
• The quantum classifier required significantly fewer parameters—just 2 gates—compared to 5 parameters in logistic regression and 110 support vectors in SVM, demonstrating strong parameter efficiency.
• The adaptive learning target mechanism improved training stability and enabled the agent to achieve higher accuracy without relying on a high initial target accuracy.
• The framework produced shallow circuits with minimal gate depth, making them suitable for execution on current noisy quantum hardware.
• Numerical simulations confirmed that the RL agent could discover effective, task-specific VQC architectures without prior ansatz design, reducing the need for expert intervention.