[Paper Review] How Descriptive are GMRES Convergence Bounds?

Mark Embree|arXiv (Cornell University)|Sep 2, 2022

Matrix Theory and Algorithms41 citations

TL;DR

The paper compares three standard GMRES convergence bounds—eigenvalues with eigenvector conditioning, field of values, and pseudospectra—and analyzes their descriptive power for nonnormal matrices, including localization via spectral projectors and practical estimation from Arnoldi.

ABSTRACT

GMRES is a popular Krylov subspace method for solving linear systems of equations involving a general non-Hermitian coefficient matrix. The conventional bounds on GMRES convergence involve polynomial approximation problems in the complex plane. Three popular approaches pose this approximation problem on the spectrum, the field of values, or pseudospectra of the coefficient matrix. We analyze and compare these bounds, illustrating with six examples the success and failure of each. When the matrix departs from normality due only to a low-dimensional invariant subspace, we discuss how these bounds can be adapted to exploit this structure. Since the Arnoldi process that underpins GMRES provides approximations to the pseudospectra, one can estimate the GMRES convergence bounds as an iteration proceeds.

Motivation & Objective

• Assess how well three common GMRES bounds describe convergence for nonnormal matrices.
• Compare bounds based on eigenvalues/eigenvectors, field of values, and pseudospectra across multiple examples.
• Develop localization techniques using spectral projectors to tighten bounds.
• Propose practical bounds and estimation approaches that can be used during GMRES iterations.

Proposed method

• Review and formulate three standard GMRES bounds: (EV) using eigenvalues with eigenvector condition numbers, (FOV) based on the field of values, and (PSA) based on pseudospectra.
• Introduce spectral projectors to localize bounds to subsets of the spectrum, yielding (EV'), (FOV'), and (PSA') variants.
• Use theoretical results (Theorem 1 and Corollaries 2–6) to show how conditioning of individual eigenvalues and invariant subspaces affect bounds.
• Discuss the CG region (Omega_CG) as a method to handle 0 in W(A) and obtain alternative bounds (CG).
• Describe practical strategies to estimate bounds during GMRES via approximate pseudospectra from the Arnoldi process (Section 4).
• Clarify how asymptotic convergence rates relate to minimax polynomial approximation over sets like sigma(A), W(A), and sigma_epsilon(A), using conformal mapping and related bounds.

Experimental results

Research questions

• RQ1 How descriptive are the standard GMRES bounds based on eigenvalues, field of values, and pseudospectra for nonnormal matrices?
• RQ2 Can localization via spectral projectors yield sharper, more accurate GMRES bounds?
• RQ3 How can one estimate GMRES convergence bounds during iterations using information from Arnoldi (pseudospectra approximations)?
• RQ4 What are the limitations of each bound (EV, FOV, PSA) and in which scenarios do they fail or succeed?
• RQ5 How does the presence of 0 in the field of values or pseudospectrum affect bound usefulness and how can region-based approaches (CG) mitigate this?

Key findings

• Each standard bound (EV, FOV, PSA) can significantly overestimate GMRES convergence for nonnormal matrices, but all can be descriptive in many cases.
• Spectral projector localization (EV', FOV', PSA') sharpens bounds by decoupling eigenvalue groups and incorporating projector norms.
• Pseudospectral bounds can capture multiple phases of convergence by sampling over a range of epsilon values, reflecting different convergence regimes.
• The CG region (Omega_CG) provides a way to handle cases where 0 lies in the field of values, yielding useful bounds when W(A) contains 0.
• Eigenvalue conditioning of simple eigenvalues (kappa(lambda)) leads to tighter bounds (EV') than the classical EV bound in certain nonnormal settings.
• Bounds based on W(A) can miss convergence behavior when eigenvalues are distributed non-uniformly or when outliers drive rapid initial convergence, highlighting limitations of FOV.
• Computing pseudospectra can be expensive, but approximate pseudospectra derived from the Arnoldi process offer a practical estimation approach (Section 4).