[论文解读] Reinforcement learning for optimization of variational quantum circuit architectures

Mateusz Ostaszewski, Lea M. Trenkwalder+3 more|arXiv (Cornell University)|Mar 30, 2021

Quantum Computing Algorithms and Architecture参考文献 40被引用 54

一句话总结

本论文提出一种结合内在课程学习的深度强化学习框架，用于自动构建浅层、门控高效的变分量子电路（VQE）ansätze，在LiH上实现化学精确度，同时最小化电路深度。与HE和UCCSD基线相比具有竞争力，并通过移动阈值训练策略扩展到更大规模的量子比特数。

ABSTRACT

The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.

研究动机与目标

• 动机：在满足NISQ约束的情况下，解决变分量子本征值求解器（VQE）中的体系结构优化需求。

提出的方法

• 将ansatz 构建形式化为一个强化学习任务，使用 DDQN 和离散动作空间来执行门插入。
• 将电路状态表示为门-层条目的有序列表；通过独立于 RL 智能体的经典子程序（COBYLA 或 Rotosolve）优化旋转角度。
• 使用奖励结构，强力激励达到化学精确度，同时惩罚过深的电路深度（最大层数 L），并奖励能量改进。

实验结果

研究问题

• RQ1 Can a reinforcement learning agent autonomously construct compact, accurate VQE ansätze for LiH under NISQ constraints?
• RQ2 How does intrinsic curriculum learning influence the agent's ability to reach chemical accuracy with shallow circuits?
• RQ3 What is the impact of global versus local angle optimization on circuit depth and gate count?
• RQ4 How does the moving-threshold approach perform when exact ground-state energy is unavailable or approximated?
• RQ5 How does the RL-derived architecture compare to HE and UCCSD baselines in depth and gate efficiency?

主要发现

方法	平均深度	最小深度	平均门数	最小门数
RL global COBYLA	14	12	36	29
HE	17	17	63	63
UCCSD	377	377	610	610

• The RL approach yields chemical accuracy with shallower circuits than HE and UCCSD in most cases for 4-qubit LiH across bond distances 1.2Å, 2.2Å, and 3.4Å.
• With 6-qubit LiH at 2.2Å, the moving-threshold curriculum RL achieves chemical accuracy in 2 of 10 trials, with avg depth 14 and min depth 12, avg gates 36 and min gates 29 under global COBYLA optimization.
• For the 6-qubit case, the RL method generates circuits roughly five times shallower than the UCCSD baseline reported in the comparison, and shallower than HE in average cases.
• COBYLA generally outperforms Rotosolve in producing shallower circuits that reach chemical accuracy under the RL framework.
• The intrinsically motivated moving-threshold curriculum enables learning without requiring prior knowledge of the exact ground-state energy, even when using a lower bound proxy.