[Paper Review] Neural Quantum States of frustrated magnets: generalization and sign structure

Tom Westerhout, Nikita Astrakhantsev+3 more|arXiv (Cornell University)|Jul 18, 2019

Quantum many-body systems50 references17 citations

TL;DR

This paper investigates the generalization and sign structure challenges in using Neural Quantum States (NQS) for frustrated quantum magnets. By training neural networks on limited exact diagonalization data, the study finds that generalization performance drops sharply with increasing frustration, and learning sign structures is significantly harder than learning amplitudes—highlighting generalization, not expressivity, as the key bottleneck for NQS in realistic models.

ABSTRACT

Neural quantum states (NQS) attract a lot of attention due to their potential to serve as a very expressive variational ansatz for quantum many-body systems. Here we study the main factors governing the applicability of NQS to frustrated magnets by training neural networks to approximate ground states of several moderately-sized Hamiltonians using the corresponding wavefunction structure on a small subset of the Hilbert space basis as training dataset. We notice that generalization quality, i.e. the ability to learn from a limited number of samples and correctly approximate the target state on the rest of the space, drops abruptly when frustration is increased. We also show that learning the sign structure is considerably more difficult than learning amplitudes. Finally, we conclude that the main issue to be addressed at this stage, in order to use the method of NQS for simulating realistic models, is that of generalization rather than expressibility.

Motivation & Objective

• To assess the applicability of Neural Quantum States (NQS) to frustrated quantum magnets with increasing spin frustration.
• To investigate whether neural networks can generalize well from limited training data sampled from the Hilbert space basis.
• To compare the difficulty of learning wavefunction amplitudes versus sign structures in NQS.
• To determine whether generalization or expressivity is the primary obstacle in using NQS for realistic many-body quantum systems.

Proposed method

• Training two separate neural networks: one to predict the sign structure of the wavefunction and another to predict the amplitudes, using exact diagonalization data as labels.
• Sampling training and validation datasets from the Hilbert space basis according to the probability distribution $ P(i) = |\psi_i|^2 / \sum_j |\psi_j|^2 $, ensuring high-amplitude configurations are prioritized.
• Using binary cross-entropy loss for the sign classifier and mean squared error on log-amplitudes for the amplitude network.
• Employing Adam optimization with learning rates of $ 10^{-4} $ to $ 10^{-3} $, and regularization via early stopping, dropout, and $ L_2 $-regularization.
• Evaluating model performance via overlap between the exact ground state and the NQS trial state.
• Reproducing results using code available on GitHub, with hyperparameters tuned for optimal performance.

Experimental results

Research questions

• RQ1 How does increasing frustration in quantum magnets affect the generalization ability of neural networks in approximating ground states?
• RQ2 To what extent is learning the sign structure of the wavefunction more difficult than learning the amplitude structure in NQS?
• RQ3 Does the performance of NQS in frustrated systems depend more on expressivity or generalization from limited data?
• RQ4 Can neural networks trained on a small subset of the Hilbert space basis accurately predict the full wavefunction structure in frustrated magnets?

Key findings

• Generalization performance of NQS drops abruptly with increasing frustration, indicating a sharp threshold beyond which limited data become insufficient.
• Learning the sign structure of the wavefunction is significantly more difficult than learning amplitudes, as evidenced by higher error rates and slower convergence.
• The overlap between the exact ground state and the NQS trial state remains high only when both amplitude and sign structures are accurately learned, with sign errors having a disproportionate impact.
• The study identifies generalization, not expressivity, as the primary challenge for applying NQS to realistic frustrated quantum models.
• Hyperparameter tuning, especially batch size, has a non-monotonic effect on the required training fraction $ \varepsilon_{\text{train}} $, with suboptimal choices increasing the required data by up to an order of magnitude.