[Paper Review] Empowering deep neural quantum states through efficient optimization

Ao Chen, Markus Heyl|arXiv (Cornell University)|Feb 3, 2023

Quantum many-body systems56 references13 citations

TL;DR

The paper introduces minimum-step stochastic reconfiguration (MinSR) to train deep neural quantum states (NQS) with orders of magnitude less cost, enabling machine-precision ground-state energies for 2D Heisenberg J1-J2 models.

ABSTRACT

Computing the ground state of interacting quantum matter is a long-standing challenge, especially for complex two-dimensional systems. Recent developments have highlighted the potential of neural quantum states to solve the quantum many-body problem by encoding the many-body wavefunction into artificial neural networks. However, this method has faced the critical limitation that existing optimization algorithms are not suitable for training modern large-scale deep network architectures. Here, we introduce a minimum-step stochastic-reconfiguration optimization algorithm, which allows us to train deep neural quantum states with up to $10^6$ parameters. We demonstrate our method for paradigmatic frustrated spin-1/2 models on square and triangular lattices, for which our trained deep networks approach machine precision and yield improved variational energies compared to existing results. Equipped with our optimization algorithm, we find numerical evidence for gapless quantum-spin-liquid phases in the considered models, an open question to date. We present a method that captures the emergent complexity in quantum many-body problems through the expressive power of large-scale artificial neural networks.

Motivation & Objective

• Motivate and address the optimization bottleneck in neural quantum states for large-scale deep networks.
• Introduce the MinSR algorithm to reduce SR complexity from cubic to near linear in network size.
• Demonstrate training of very deep NQS (up to 64 layers, >10^5 parameters) on 2D spin models.
• Evaluate performance against existing SR/SGD methods and benchmark against known results for J1-J2 models.

Proposed method

• Formulate the SR update as solving a linear equation that projects imaginary-time evolution onto the variational manifold.
• Introduce the neural tangent kernel T = O O^† which shares nonzero eigenvalues with the quantum metric S but has size N_s × N_s.
• Derive the MinSR solution δθ = O^† T^{-1} ε with T = O O^†, reducing complexity to O(N_p N_s^2 + N_s^3).
• Use a minimum-norm (minimum-step) criterion to select the unique δθ among underdetermined solutions, improving stability.
• Apply regularization and Monte-Carlo sampling to compute O and ε from variational Monte Carlo.
• Benchmark on spin-1/2 Heisenberg J1-J2 models to demonstrate accuracy and scalability.

Experimental results

Research questions

• RQ1 Can MinSR achieve comparable variational energies to SR while drastically reducing computational cost?
• RQ2 How does MinSR scale with network depth and parameter count for large NQS architectures?
• RQ3 What are the accuracy limits of deep NQS trained with MinSR when approaching machine precision on modern hardware?
• RQ4 Can deep NQS trained with MinSR outperform existing methods for frustrated two-dimensional quantum magnets like the J1-J2 model?

Key findings

• MinSR reduces the optimization cost to leading O(N_p) for deep networks with N_p parameters, compared to O(N_p^3) for traditional SR.
• MinSR enables training of neural networks up to 64 layers and over 10^5 parameters on 2D Heisenberg J1-J2 models.
• Deep NQS trained with MinSR achieve variational energies that surpass conventional numerical approaches on 16×16 lattices for the J1-J2 model.
• The ground-state results approach different levels of machine precision on modern GPUs and TPUs, limited mainly by device numerical precision.
• For non-frustrated Heisenberg models, MinSR yields variational energies surpassing previous NQS results by large margins, with near-zero sign-structure errors in many cases.
• For the frustrated J1-J2 model at J2/J1 = 0.5 on 16×16 lattices, MinSR attains the best reported variational energy among NQS methods.