[Paper Review] Convergence analysis of direct minimization and self-consistent iterations

Éric Cancès, Gaspard Kemlin+1 more|arXiv (Cornell University)|Apr 20, 2020

Physics of Superconductivity and MagnetismPhysics and Astronomy68 references51 citations

TL;DR

This paper provides a rigorous convergence analysis of two fundamental algorithms for solving subspace optimization problems: damped self-consistent field (SCF) iterations and fixed-step gradient descent. It derives asymptotic convergence rates dependent on the spectral gap and problem condition, revealing that SCF methods can exhibit chaotic behavior for non-quadratic functionals, while gradient descent shows predictable convergence tied to the spectral properties of the Hessian operator.

ABSTRACT

This article is concerned with the numerical solution of subspace optimization problems, consisting of minimizing a smooth functional over the set of orthogonal projectors of fixed rank. Such problems are encountered in particular in electronic structure calculation (Hartree-Fock and Kohn-Sham Density Functional Theory -DFT- models). We compare from a numerical analysis perspective two simple representatives, the damped self-consistent field (SCF) iterations and the gradient descent algorithm, of the two classes of methods competing in the field: SCF and direct minimization methods. We derive asymptotic rates of convergence for these algorithms and analyze their dependence on the spectral gap and other properties of the problem. Our theoretical results are complemented by numerical simulations on a variety of examples, from toy models with tunable parameters to realistic Kohn-Sham computations. We also provide an example of chaotic behavior of the simple SCF iterations for a nonquadratic functional.

Motivation & Objective

• To compare the convergence behavior of damped SCF and gradient descent algorithms for subspace optimization problems arising in electronic structure theory.
• To analyze the asymptotic convergence rates of these two algorithms in terms of spectral properties of the problem.
• To identify conditions under which SCF iterations may fail to converge, including chaotic dynamics for non-quadratic functionals.
• To establish theoretical foundations for understanding the performance differences between SCF and direct minimization methods.
• To provide a basis for improving practical algorithms by analyzing their simplest representatives.

Proposed method

• Analyzes the subspace optimization problem min_P E(P) over rank-N orthogonal projectors, relevant to Hartree-Fock and Kohn-Sham DFT.
• Derives local convergence rates for damped SCF and fixed-step gradient descent using spectral analysis of operators of the form 1−βJ.
• Identifies the spectral gap between the Nth and (N+1)st eigenvalues of the effective Hamiltonian as a key determinant of convergence speed.
• Uses Riemannian optimization theory on the Grassmann manifold to model the set of rank-N projectors.
• Performs numerical simulations on toy models with tunable gaps and realistic Kohn-Sham systems to validate theoretical findings.
• Demonstrates chaotic behavior in SCF iterations for non-quadratic functionals, complementing prior results.

Experimental results

Research questions

• RQ1 How do the asymptotic convergence rates of damped SCF and gradient descent algorithms depend on the spectral gap of the effective Hamiltonian?
• RQ2 Under what conditions does the simple damped SCF iteration fail to converge, and can this be linked to non-quadratic functionals?
• RQ3 Why do direct minimization methods like gradient descent exhibit more predictable convergence than SCF methods in certain regimes?
• RQ4 Can the damped SCF algorithm be interpreted as a matrix splitting of the fixed-step gradient descent method?
• RQ5 What role does the Aufbau principle play in the convergence and uniqueness of solutions for these algorithms?

Key findings

• The convergence rate of both damped SCF and gradient descent algorithms is governed by the spectral radius of operators involving the Hessian and effective Hamiltonian, with the spectral gap between the Nth and (N+1)st eigenvalues being a critical factor.
• For non-quadratic functionals, SCF iterations can exhibit chaotic behavior, which is analytically demonstrated and numerically verified in the paper.
• The damped SCF algorithm can be viewed as a matrix splitting of the fixed-step gradient descent method, providing a theoretical link between the two classes.
• In the case of degenerate eigenvalues (εN = εN+1), the minimizer of the energy functional may not be unique, and the Aufbau principle may fail, affecting convergence.
• When the spectral gap is large, both algorithms converge linearly, but the convergence rate of SCF is generally slower than that of gradient descent for the same step size.
• The analysis reveals that preconditioning is essential for extending convergence theory to infinite-dimensional settings, and this remains an open area for further research.