[논문 리뷰] Reinforcement learning for optimization of variational quantum circuit architectures

Mateusz Ostaszewski, Lea M. Trenkwalder외 3명|arXiv (Cornell University)|2021. 03. 30.

Quantum Computing Algorithms and Architecture참고 문헌 40인용 수 54

한 줄 요약

본 논문은 고유의 커리큘럼 학습을 갖춘 심층 강화 학습 프레임워크를 통해 얕고 게이트 효율적인 변분 양자 회로(VQE) 애당스트를 자동으로 구성하고, LiH에 대해 화학 정확도를 달성하면서 회로 깊이를 최소화합니다. HE 및 UCCSD 기준선과 비교해 우수한 성능을 보이며, 이동 임계값 학습 전략을 통해 더 많은 큐빗에서 확장합니다.

ABSTRACT

The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.

연구 동기 및 목표

• NISQ 제약 하에서 VQE(Variational Quantum Eigensolvers)의 아키텍처 최적화 필요성을 동기 부여하고 해결하는 것.

제안 방법

• Ansatz 구성을 DDQN과 게이트 삽입에 대한 이산 행동 공간을 갖는 강화학습 문제로 형식화한다.
• 회로 상태를 게이트-레이어 항목의 순차적 목록으로 표현하고, 회전 각은 RL 에이전트와 분리된 고전적 서브루틴(COBYLA 또는 Rotosolve)을 통해 최적화한다.
• 화학 정확도 달성을 강하게 유도하고 과도한 깊이(최대 레이어 수 L)를 페널티하며 에너지 개선에 보상하는 보상 구조를 사용한다.

실험 결과

연구 질문

• RQ1 Can a reinforcement learning agent autonomously construct compact, accurate VQE ansätze for LiH under NISQ constraints?
• RQ2 How does intrinsic curriculum learning influence the agent's ability to reach chemical accuracy with shallow circuits?
• RQ3 What is the impact of global versus local angle optimization on circuit depth and gate count?
• RQ4 How does the moving-threshold approach perform when exact ground-state energy is unavailable or approximated?
• RQ5 How does the RL-derived architecture compare to HE and UCCSD baselines in depth and gate efficiency?

주요 결과

• The RL approach yields chemical accuracy with shallower circuits than HE and UCCSD in most cases for 4-qubit LiH across bond distances 1.2Å, 2.2Å, and 3.4Å.
• With 6-qubit LiH at 2.2Å, the moving-threshold curriculum RL achieves chemical accuracy in 2 of 10 trials, with avg depth 14 and min depth 12, avg gates 36 and min gates 29 under global COBYLA optimization.
• For the 6-qubit case, the RL method generates circuits roughly five times shallower than the UCCSD baseline reported in the comparison, and shallower than HE in average cases.
• COBYLA generally outperforms Rotosolve in producing shallower circuits that reach chemical accuracy under the RL framework.
• The intrinsically motivated moving-threshold curriculum enables learning without requiring prior knowledge of the exact ground-state energy, even when using a lower bound proxy.