[论文解读] Is attention all you need to solve the correlated electron problem?

Max Geier, Khachatur G. Nazaryan+2 more|ArXiv.org|Feb 7, 2025

Electron and X-Ray Spectroscopy Techniques被引用 3

一句话总结

这篇论文在变分蒙特卡洛（VMC）中使用具有大量参数的自注意力神经网络作为变分波函数来求解莫尔量子材料中相互作用电子的问题，发现参数量大致随 N^2 增长，并在 Hartree-Fock 和 BP-ED 之上取得准确结果。

ABSTRACT

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.

研究动机与目标

• Motivate the use of neural network variational Monte Carlo (NN-VMC) for solving the many-electron problem in solids.
• Introduce a self-attention based neural network wavefunction that generates generalized Slater determinants.
• Demonstrate accuracy and efficiency of the attention-based ansatz on moiré semiconductor systems.
• Assess how the number of variational parameters scales with system size and compare to traditional methods.

提出的方法

• Construct a SlaterNet to generate unrestricted Hartree-Fock-like orbitals via a deep feed-forward network.
• Incorporate self-attention to mix particle streams and capture correlations, producing correlated orbitals.
• Form a multi-determinant wavefunction as a sum of determinants built from correlated orbitals (Psi = sum det()).
• Optimize the variational parameters using variational Monte Carlo with the energy objective and natural gradient (KFAC approximation).
• Benchmark against Hartree-Fock and band-projected exact diagonalization for WSe2/WS2 moiré heterobilayers.

Figure 1: Architecture of the neural network wavefunction ansatz. SlaterNet : Multilayer perceptron neural network generates one-body orbitals to approximate general single Slater determinant wavefunctions. Psi-Solid : Self-attention neural network for solids based on Psi-Former [ 11 ] capturing cor

实验结果

研究问题

• RQ1 Can a self-attention based neural network wavefunction serve as a universal, scalable ansatz for interacting electrons in solids?
• RQ2 How does the required number of variational parameters scale with the number of electrons in a moiré material system?
• RQ3 How does the NN-VMC approach compare to conventional methods (HF, BP-ED) in accuracy for moiré semiconductor Hamiltonians?
• RQ4 What is the performance of a self-attention NN wavefunction in capturing electron correlations in two-dimensional moiré lattices?
• RQ5 Does the attention-based ansatz remain accurate as system size grows toward large-scale simulations?

主要发现

• The self-attention NN wavefunction provides an accurate, efficient, and unbiased solution to the correlated electron problem in the studied moiré system.
• For the moiré Hamiltonian, the saturation energy can be reached with a number of parameters N_par that scales approximately as N^2 (N_par* ≈ 320 × N^2.1).
• Self-attention based wavefunctions outperform band-projected exact diagonalization when five bands are included in benchmarks.
• The approach remains robust as system size increases, suggesting scalability to larger, more realistic solids.
• The method achieves lower energies than BP-ED for small systems and captures correlation effects beyond Hartree-Fock.

Figure 2: Building blocks of variational Monte Carlo. In the Monte Carlo algorithm, the wavefunction ansatz $\Psi_{\theta}$ is constructed and sampled to efficiently evaluate the optimization goal $L[\theta]:=\langle\Psi_{\theta}|\hat{H}|\Psi_{\theta}\rangle$ of minimizing the energy. Accordingly up