[Paper Review] Compiler Optimization for Quantum Computing Using Reinforcement Learning

Nils Quetschlich, Lukas Burgholzer+1 more|arXiv (Cornell University)|Dec 8, 2022

Quantum Computing Algorithms and Architecture35 references4 citations

TL;DR

This paper proposes a reinforcement learning framework that models quantum circuit compilation as a Markov Decision Process (MDP), enabling automated learning of optimal optimization pass sequences. By unifying diverse compilation passes from Qiskit and TKET through a standardized interface and constraints, the framework achieves superior performance—outperforming both individual compilers in 73% of cases on expected fidelity, 84% on critical depth, and 75% on combined metrics.

ABSTRACT

Any quantum computing application, once encoded as a quantum circuit, must be compiled before being executable on a quantum computer. Similar to classical compilation, quantum compilation is a sequential process with many compilation steps and numerous possible optimization passes. Despite the similarities, the development of compilers for quantum computing is still in its infancy -- lacking mutual consolidation on the best sequence of passes, compatibility, adaptability, and flexibility. In this work, we take advantage of decades of classical compiler optimization and propose a reinforcement learning framework for developing optimized quantum circuit compilation flows. Through distinct constraints and a unifying interface, the framework supports the combination of techniques from different compilers and optimization tools in a single compilation flow. Experimental evaluations show that the proposed framework -- set up with a selection of compilation passes from IBM's Qiskit and Quantinuum's TKET -- significantly outperforms both individual compilers in 73% of cases regarding the expected fidelity. The framework is available on GitHub (https://github.com/cda-tum/MQTPredictor) as part of the Munich Quantum Toolkit (MQT).

Motivation & Objective

• Address the lack of standardized, flexible, and adaptive quantum compiler flows that hinder performance and interoperability.
• Overcome vendor lock-in and slow adoption of new optimization techniques in major quantum SDKs like Qiskit and TKET.
• Enable dynamic, customizable optimization goals by modeling compilation as a learnable process rather than fixed optimization levels.
• Leverage decades of classical compiler research—especially phase ordering via reinforcement learning—adapted to the quantum domain.
• Develop a platform-agnostic, extensible framework that integrates optimization passes from multiple quantum compilers and tools.

Proposed method

• Model the quantum circuit compilation process as a Markov Decision Process (MDP), where states represent circuit configurations, actions are application of optimization passes, and rewards are based on target metrics like fidelity and critical depth.
• Define a unifying interface and distinct constraints to ensure compatibility and correctness when combining compilation passes from different tools (e.g., Qiskit and TKET).
• Use deep reinforcement learning (specifically, a policy gradient method) to train an agent to select the optimal sequence of optimization passes for a given input circuit.
• Design a reward function that combines expected fidelity and critical depth to guide the agent toward high-quality, low-depth, and accurate circuits.
• Implement the framework as part of the Munich Quantum Toolkit (MQT), exposing it via GitHub for extensibility and integration with existing tools.
• Support incremental extension by allowing new passes to be added as long as they conform to the interface and constraint checks.

Experimental results

Research questions

• RQ1 Can reinforcement learning effectively learn optimal sequences of quantum circuit optimization passes across heterogeneous compiler tools?
• RQ2 To what extent can a unified framework combining passes from Qiskit and TKET outperform the native compilation flows of these tools in terms of fidelity and circuit depth?
• RQ3 How does the proposed framework handle compatibility and correctness when integrating diverse optimization techniques from different quantum software stacks?
• RQ4 Can the framework adapt to user-defined optimization goals, such as minimizing circuit depth or maximizing fidelity, without hardcoding fixed optimization levels?
• RQ5 How scalable and generalizable is the approach across different quantum circuits and hardware targets?

Key findings

• The proposed reinforcement learning framework outperforms IBM’s Qiskit and Quantinuum’s TKET in 73% of test cases with respect to expected fidelity.
• The framework achieves superior results in 84% of cases on critical depth, indicating better circuit depth reduction.
• When combining fidelity and depth as a joint metric, the framework outperforms both individual compilers in 75% of cases.
• The framework successfully integrates optimization passes from Qiskit and TKET into a single, coherent compilation flow through a unifying interface and constraint system.
• The approach enables dynamic, customizable optimization flows that adapt to user-defined goals, overcoming the limitations of fixed optimization levels in existing SDKs.
• The framework is extensible and platform-agnostic, with code available on GitHub as part of the Munich Quantum Toolkit (MQT), facilitating adoption and further research.