[Paper Review] Adaptive Step Size Strategy for Orthogonality Constrained Line Search Methods

Xiaoying Dai, Liwei Zhang+1 more|arXiv (Cornell University)|Jun 7, 2019

Matrix Theory and Algorithms32 references4 citations

TL;DR

This paper proposes an adaptive step size strategy for orthogonality constrained line search methods that eliminates the need for backtracking while maintaining convergence. The method achieves faster convergence and reduced computational cost in electronic structure calculations, outperforming traditional backtracking-based approaches in both iteration count and runtime across multiple molecular systems.

ABSTRACT

In this paper, we propose an adaptive step size strategy for a class of line search methods for orthogonality constrained minimization problems, which avoids the classic backtracking procedure. We prove the convergence of the line search methods equipped with our adaptive step size strategy under some mild assumptions.We then apply the adaptive algorithm to electronic structure calculations, which show that our strategy is efficient and recommended.

Motivation & Objective

• To develop a step size strategy for orthogonality constrained minimization that avoids the computationally expensive backtracking procedure.
• To ensure global convergence of line search methods under mild assumptions without relying on backtracking.
• To improve computational efficiency in solving Kohn-Sham DFT problems for electronic structure calculations.
• To provide a robust, adaptive alternative to Armijo-type backtracking that reduces iteration count and runtime.

Proposed method

• Introduces an adaptive step size strategy that dynamically adjusts the step size based on local descent conditions, avoiding repeated trial points and function evaluations.
• Establishes theoretical convergence under mild assumptions, including bounded gradient and Lipschitz continuity of the gradient.
• Applies the adaptive strategy to conjugate gradient and gradient-type methods on the Stiefel manifold.
• Employs a retraction-based line search framework with a non-monotone Armijo-type condition adapted for adaptive step size selection.
• Uses a uniform algorithmic framework applicable to various line search methods, including CG and gradient descent, under orthogonality constraints.
• Validates the strategy numerically on Kohn-Sham energy minimization problems using real molecular systems.

Experimental results

Research questions

• RQ1 Can an adaptive step size strategy be designed to replace backtracking in orthogonality constrained line search methods without sacrificing convergence?
• RQ2 Does the proposed adaptive strategy reduce computational cost compared to backtracking-based methods in electronic structure calculations?
• RQ3 How does the adaptive strategy perform in terms of convergence speed and final accuracy across different molecular systems?
• RQ4 Can the adaptive method maintain global convergence under weaker assumptions than those required by classical backtracking methods?

Key findings

• The adaptive step size strategy achieves global convergence under mild assumptions, including bounded gradient and Lipschitz continuity of the gradient, without requiring backtracking.
• For benzene (C6H6), the adaptive method reduced computational time from 31.39s (backtracking) to 11.36s, with fewer iterations (334 vs. 625).
• For fullerene (C60), the adaptive method reduced computational time from 6775.76s (backtracking) to 945.60s, despite a higher per-iteration cost.
• The backtracking-free version (GM-QR-noBack) failed to converge within 30,000 iterations for alanine and C120, while the adaptive method converged in 3,376 and 7,929 iterations, respectively.
• The adaptive method achieved comparable or better energy values than backtracking-based methods, with final energies matching the reference values (e.g., -3.42875137E+02 a.u. for C60).
• The adaptive strategy reduced average time per iteration (A.T.P.I.) by up to 30% compared to backtracking, despite higher per-iteration cost, due to fewer iterations.