[Paper Review] Optimizing Quantum Annealing Schedules: From Monte Carlo Tree Search to QuantumZero

Yuqin Chen, Yu Chen+3 more|arXiv (Cornell University)|Apr 6, 2020

Quantum Computing Algorithms and Architecture49 references4 citations

TL;DR

This paper proposes QuantumZero (QZero), a neural network-enhanced Monte Carlo Tree Search (MCTS) framework to automatically optimize quantum annealing schedules for adiabatic quantum computation. By leveraging MCTS and transfer learning, QZero efficiently discovers high-success-rate schedules, outperforming other reinforcement learning methods—especially when solving large sets of similar problems after pre-training.

ABSTRACT

Quantum annealing is a practical approach to execute the native instruction set of the adiabatic quantum computation model. The key of running adiabatic algorithms is to maintain a high success probability of evolving the system into the ground state of a problem-encoded Hamiltonian at the end of an annealing schedule. This is typically done by executing the adiabatic algorithm slowly to enforce adiabacity. However, properly optimized annealing schedule can accelerate the computational process. Inspired by the recent success of DeepMind's AlphaZero algorithm that can efficiently explore and find a good winning strategy from a large combinatorial search with a neural-network-assisted Monte Carlo Tree Search (MCTS), we adopt MCTS and propose a neural-network-enabled version, termed QuantumZero (QZero), to automate the design of an optimal annealing schedule in a hybrid quantum-classical framework. The flexibility of having neural networks allows us to apply transfer-learning technique to boost QZero's performance. We find both MCTS and QZero to perform very well in finding excellent annealing schedules even when the annealing time is short in the 3-SAT examples we consider in this study. We also find MCTS and QZero to be more efficient than many other leading reinforcement leanring algorithms for the task of desining annealing schedules. In particular, if there is a need to solve a large set of similar problems using a quantum annealer, QZero is the method of choice when the neural networks are first pre-trained with examples solved in the past.

Motivation & Objective

• To address the challenge of designing optimal annealing schedules that maximize the probability of reaching the ground state in quantum annealing.
• To automate the discovery of efficient annealing schedules using machine learning, reducing reliance on slow, manual tuning.
• To improve computational efficiency by accelerating the convergence of quantum annealing beyond standard adiabatic evolution.
• To enable transfer learning across similar problems, enhancing performance when solving large batches of related 3-SAT instances.
• To outperform existing reinforcement learning baselines in scheduling optimization for quantum annealers.

Proposed method

• Adopt a hybrid quantum-classical framework where classical MCTS explores the space of possible annealing schedules.
• Integrate a neural network into MCTS to guide tree exploration and value estimation, enabling efficient search in high-dimensional schedule spaces.
• Use transfer learning by pre-training the neural network on previously solved 3-SAT problems to accelerate learning on new, similar problems.
• Apply the MCTS-based QZero algorithm to iteratively refine annealing schedules based on success probability feedback from quantum annealing runs.
• Optimize the annealing schedule by balancing exploration and exploitation in the search space using tree policy and backup functions.
• Leverage the neural network's generalization to extrapolate from known good schedules to new, similar problem instances.

Experimental results

Research questions

• RQ1 Can a neural network-augmented MCTS approach effectively discover high-performance annealing schedules for quantum annealing?
• RQ2 How does QZero compare to other reinforcement learning algorithms in optimizing annealing schedules?
• RQ3 To what extent can transfer learning improve the efficiency of annealing schedule optimization across similar problems?
• RQ4 Does QZero maintain high success probability even under short annealing times?
• RQ5 Can MCTS and QZero outperform traditional adiabatic evolution in terms of computational speed and success rate?

Key findings

• QZero and MCTS both achieve high success probabilities in finding optimal annealing schedules, even with short annealing times in 3-SAT problems.
• The neural network-enhanced QZero outperforms multiple leading reinforcement learning algorithms in scheduling optimization tasks.
• Transfer learning significantly boosts QZero’s performance when solving large sets of similar problems after pre-training on prior examples.
• QZero enables faster convergence to high-quality schedules compared to standard adiabatic evolution, reducing the need for slow, brute-force tuning.
• The method demonstrates robustness and scalability in combinatorial search spaces typical of quantum annealing problems.
• Both MCTS and QZero show strong generalization, maintaining high performance across diverse 3-SAT instances with minimal retraining.