[Paper Review] How Accurate is inv(A)*b?

Alex Druinsky, Sivan Toledo|arXiv (Cornell University)|Jan 29, 2012

Matrix Theory and Algorithms8 references3 citations

TL;DR

This paper demonstrates that solving $Ax = b$ via $x = \text{inv}(A) \cdot b$ is as accurate as using backward-stable solvers like LU or QR factorization, provided the inverse is computed conditionally accurately. The key finding is that the solution is conditionally accurate and often backward stable when $b$ is not aligned with small singular subspaces of $A$, resolving a long-standing misconception in numerical linear algebra.

ABSTRACT

Several widely-used textbooks lead the reader to believe that solving a linear system of equations Ax = b by multiplying the vector b by a computed inverse inv(A) is inaccurate. Virtually all other textbooks on numerical analysis and numerical linear algebra advise against using computed inverses without stating whether this is accurate or not. In fact, under reasonable assumptions on how the inverse is computed, x = inv(A)*b is as accurate as the solution computed by the best backward-stable solvers. This fact is not new, but obviously obscure. We review the literature on the accuracy of this computation and present a self-contained numerical analysis of it.

Motivation & Objective

• To correct the widespread misconception that computing $x = \text{inv}(A) \cdot b$ is numerically inaccurate compared to $A \backslash b$.
• To provide a rigorous numerical analysis showing that $x = \text{inv}(A) \cdot b$ yields conditionally accurate solutions under standard inverse computation methods.
• To clarify when the solution is not only accurate but also backward stable, and when it is only conditionally accurate.
• To demonstrate through numerical experiments that the accuracy of $\text{inv}(A) \cdot b$ matches that of standard solvers like LU or QR, especially when $b$ is not aligned with small singular subspaces.

Proposed method

• Analyzes the forward error of $x_V = (V + \Delta)b$ where $V \approx A^{-1}$, using bounds on $\|V - A^{-1}\|$ and $\|\Delta\|$.
• Applies Wilkinson's tighter error bound, showing that the error is $O(\kappa(A)\epsilon_{\text{machine}})\|x\|$, matching backward-stable solvers.
• Introduces the concept of conditional accuracy and backward stability, distinguishing cases where $x_V$ is both accurate and backward stable.
• Uses SVD-based construction of $A$ and $A^{-1}$ to generate controlled test matrices with known condition numbers and singular values.
• Compares $\text{inv}(A) \cdot b$ with $A \backslash b$ using MATLAB’s backslash and SVD-based exact solution, measuring residual and error norms.
• Visualizes the structure of $\Gamma = V - A^{-1}$ to show that its rows are nearly orthogonal to left singular vectors of $A$ associated with large singular values.

Experimental results

Research questions

• RQ1 Is $x = \text{inv}(A) \cdot b$ as accurate as solutions from backward-stable solvers like LU or QR factorization?
• RQ2 Under what conditions is $x = \text{inv}(A) \cdot b$ not only accurate but also backward stable?
• RQ3 Why do many textbooks incorrectly claim that $\text{inv}(A) \cdot b$ is less accurate than $A \backslash b$?
• RQ4 What role does the alignment of $b$ with the left singular subspaces of $A$ play in the accuracy and stability of $\text{inv}(A) \cdot b$?
• RQ5 Can a computed inverse still yield a conditionally accurate solution even if it is not forward-accurate?

Key findings

• The forward error of $x = \text{inv}(A) \cdot b$ is bounded by $O(\kappa^2(A)\epsilon_{\text{machine}})\|x\|$, but this bound is loose; the actual error is $O(\kappa(A)\epsilon_{\text{machine}})\|x\|$, matching backward-stable solvers.
• When $b$ is not aligned with the left singular subspaces corresponding to small singular values, $x = \text{inv}(A) \cdot b$ is both conditionally accurate and backward stable.
• Numerical experiments show that $\text{inv}(A) \cdot b$ achieves a residual norm of $8.8 \times 10^{-16}$, comparable to MATLAB’s backslash operator ($4.1 \times 10^{-9}$ relative error).
• The solution remains conditionally accurate even when $b$ is nearly orthogonal to small singular subspaces, with relative error $3.1 \times 10^{-9}$, but backward error increases to $2.1 \times 10^{-10}$.
• A random error matrix added to $V$ results in a solution with $83.5\%$ relative error, showing that structure in $\Gamma = V - A^{-1}$ is essential for accuracy.
• The paper confirms that $\text{inv}(A) \cdot b$ is not inherently inaccurate—its accuracy depends on the structure of the inverse error, not on the method itself.