[Paper Review] Smoothed Analysis of the Condition Numbers and Growth Factors of Matrices

Arvind Sankar, Daniel A. Spielman+1 more|ArXiv.org|Oct 12, 2003

Random Matrices and Applications14 references4 citations

TL;DR

This paper introduces a smoothed analysis framework to show that random perturbations of arbitrary matrices are highly unlikely to have large condition numbers or high growth factors under Gaussian elimination without pivoting. The key result is that only logarithmic precision—specifically O(b + log n) bits—is typically needed for accurate solution of linear systems, explaining the practical success of Gaussian elimination despite worst-case theoretical limitations.

ABSTRACT

Let $\orig{A}$ be any matrix and let $A$ be a slight random perturbation of $\orig{A}$. We prove that it is unlikely that $A$ has large condition number. Using this result, we prove it is unlikely that $A$ has large growth factor under Gaussian elimination without pivoting. By combining these results, we bound the smoothed precision needed by Gaussian elimination without pivoting. Our results improve the average-case analysis of Gaussian elimination without pivoting performed by Yeung and Chan (SIAM J. Matrix Anal. Appl., 1997).

Motivation & Objective

• To explain why Gaussian elimination without pivoting performs well in practice despite poor worst-case complexity.
• To analyze the likelihood of large condition numbers and growth factors in matrices under small random perturbations.
• To establish that smoothed precision requirements for solving linear systems are logarithmic, resolving a gap between theory and practice.
• To extend prior average-case analyses by incorporating noise, making results more robust to real-world input variations.
• To lay foundational results for smoothed analysis of Gaussian elimination with partial or complete pivoting.

Proposed method

• Uses smoothed analysis—a hybrid of worst-case and average-case analysis—by studying the expected performance of algorithms on slightly perturbed inputs.
• Analyzes the condition number $κ(\mathbf{A}) = \|\mathbf{A}\|_2 \|\mathbf{A}^{-1}\|_2$ under Gaussian perturbations of an arbitrary matrix $\bar{\mathbf{A}}$.
• Proves that the probability of $\kappa(\mathbf{A}) \geq x$ is bounded by $O(n / (x\sigma))$ when $\|\bar{\mathbf{A}}\|_2 \leq \sqrt{n}$, under Gaussian noise of variance $\sigma^2 \leq 1$.
• Links condition number bounds to growth factors in Gaussian elimination without pivoting, showing that large growth factors are unlikely under perturbation.
• Derives that $O(b + \log n)$ bits of precision suffice for solving $\mathbf{A}\mathbf{x} = \mathbf{b}$ with $b$-bit accuracy, due to low likelihood of large intermediate entries.
• Considers zero-preserving perturbations and symmetric matrix structures, showing that certain ill-conditioned matrices remain problematic under such constraints.

Experimental results

Research questions

• RQ1 What is the probability that a slightly perturbed matrix has a large condition number?
• RQ2 How likely is it that Gaussian elimination without pivoting produces a large growth factor on a perturbed matrix?
• RQ3 Can smoothed analysis explain the practical robustness of Gaussian elimination despite its worst-case complexity?
• RQ4 Do zero-preserving perturbations preserve ill-conditioning or high growth factors in symmetric or non-symmetric matrices?
• RQ5 What are the smoothed bounds on growth factors under partial or complete pivoting, and how do they compare to no pivoting?

Key findings

• The probability that a Gaussian perturbation of an arbitrary matrix $\bar{\mathbf{A}}$ has condition number $\kappa(\mathbf{A}) \geq x$ is bounded by $O(n / (x\sigma))$ when $\|\bar{\mathbf{A}}\|_2 \leq \sqrt{n}$ and $\sigma^2 \leq 1$.
• For matrices with $\|\bar{\mathbf{A}}\|_2 \leq \sqrt{n}$, the smoothed condition number is unlikely to exceed $x$ with probability $O(n \log n / (x\sigma))$.
• Large growth factors under Gaussian elimination without pivoting are unlikely in perturbed matrices, due to low likelihood of large intermediate entries.
• The smoothed precision required for Gaussian elimination without pivoting is $O(b + \log n)$ bits, where $b$ is the input precision, explaining practical efficiency.
• Counterexamples show that zero-preserving perturbations can preserve ill-conditioning in non-symmetric matrices and large growth factors in structured matrices.
• Wschebor's improved bound shows $\Pr[\kappa(\mathbf{A}) \geq x] \leq O(n \log n / (x\sigma))$ under similar conditions, supporting the conjecture that $O(n / (x\sigma))$ is tight.