[Paper Review] Convergence analysis of adaptive DIIS algorithms with application to electronic ground state calculations

Maxime Chupin, Mi-Song Dupuy+2 more|arXiv (Cornell University)|Feb 28, 2020

Numerical methods for differential equations67 references22 citations

TL;DR

This paper proposes two adaptive depth strategies for Anderson–Pulay acceleration methods—restart-based and continuously adaptive—dynamically adjusting the number of stored iterates to improve convergence in self-consistent field calculations. The authors prove local superlinear convergence without requiring bounded extrapolation coefficients, and numerical results show both variants outperform fixed-depth schemes with lower average computational cost.

ABSTRACT

This paper deals with a general class of algorithms for the solution of fixed-point problems that we refer to as \\emph{Anderson--Pulay acceleration}. This family includes the DIIS technique and its variant sometimes called commutator-DIIS, both introduced by Pulay in the 1980s to accelerate the convergence of self-consistent field procedures in quantum chemistry, as well as the related Anderson acceleration which dates back to the 1960s, and the wealth of techniques they have inspired. Such methods aim at accelerating the convergence of any fixed-point iteration method by combining several iterates in order to generate the next one at each step. This extrapolation process is characterised by its \\emph{depth}, i.e. the number of previous iterates stored, which is a crucial parameter for the efficiency of the method. It is generally fixed to an empirical value. In the present work, we consider two parameter-driven mechanisms to let the depth vary along the iterations. In the first one, the depth grows until a certain nondegeneracy condition is no longer satisfied; then the stored iterates (save for the last one) are discarded and the method "restarts". In the second one, we adapt the depth continuously by eliminating at each step some of the oldest, less relevant, iterates. In an abstract and general setting, we prove under natural assumptions the local convergence and acceleration of these two adaptive Anderson--Pulay methods, and we show that one can theoretically achieve a superlinear convergence rate with each of them. We then investigate their behaviour in quantum chemistry calculations. These numerical experiments show that both adaptive variants exhibit a faster convergence than a standard fixed-depth scheme, and require on average less computational effort per iteration. This study is complemented by a review of known facts on the DIIS, in particular its link with the Anderson acceleration and some multisecant-type quasi-Newton methods.

Motivation & Objective

• To address the suboptimal performance of fixed-depth DIIS and Anderson acceleration in electronic structure calculations.
• To develop adaptive mechanisms that dynamically adjust the depth (number of stored iterates) during iterations to enhance convergence speed.
• To provide a rigorous theoretical convergence analysis for adaptive Anderson–Pulay methods under weaker assumptions than prior work.
• To demonstrate through numerical experiments that adaptive depth leads to faster convergence and reduced average computational cost compared to fixed-depth schemes.
• To clarify the theoretical link between DIIS, CDIIS, and Anderson acceleration within a unified framework.

Proposed method

• Proposes a restart-based adaptive depth mechanism that discards older iterates when a nondegeneracy condition fails, effectively restarting the history.
• Introduces a novel continuous adaptive depth strategy that removes the oldest, least relevant iterates at each step based on a new criterion.
• Analyzes both methods in an abstract fixed-point setting, proving local convergence and acceleration under weakened assumptions.
• Establishes a priori bounds on extrapolation coefficients by embedding the adaptive mechanism directly into the algorithm, avoiding reliance on post-hoc boundedness assumptions.
• Applies the methods to self-consistent field (SCF) calculations in quantum chemistry using molecular systems as test cases.
• Uses a generalized residual minimization framework to derive the extrapolation coefficients via least-squares, with depth control integrated into the coefficient computation.

Experimental results

Research questions

• RQ1 Can adaptive depth control in Anderson–Pulay acceleration methods lead to faster convergence than fixed-depth schemes in electronic structure calculations?
• RQ2 Does the proposed adaptive mechanism ensure theoretical convergence and avoid numerical instability due to linear dependency in the least-squares problem?
• RQ3 How does the performance of the adaptive methods compare to standard DIIS and Anderson acceleration in terms of convergence rate and computational cost?
• RQ4 What is the impact of the adaptive depth mechanism on the average number of stored iterates compared to conventional 'rule-of-thumb' values?
• RQ5 Can the theoretical convergence guarantees be extended to the DIIS and CDIIS methods under weaker nondegeneracy and boundedness assumptions?

Key findings

• The adaptive depth mechanisms ensure local superlinear convergence without requiring a priori boundedness of extrapolation coefficients.
• Both adaptive variants—restart-based and continuously adaptive—achieve faster convergence than their fixed-depth counterparts in electronic ground state calculations.
• The adaptive-depth variant shows particularly strong performance, suggesting it may be the most efficient in practice.
• Numerical experiments show that the average depth of stored iterates is significantly lower than the typical fixed values used in standard implementations.
• Convergence rates improve as the adaptive parameters (τ and δ) are decreased, but diminishing returns are observed beyond a certain point, with optimal performance at 10−4.
• The methods remain effective even with parameter values several orders of magnitude larger than theoretical estimates, indicating robustness in practice.