[Paper Review] A contribution to the condition number of the total least squares problem

Zhongxiao Jia, Bingyu Li|arXiv (Cornell University)|Jan 12, 2011

Statistical and numerical algorithms18 references3 citations

TL;DR

This paper presents new, computationally efficient formulas and sharp bounds for the condition number of the total least squares (TLS) problem. By leveraging singular value decomposition (SVD) and properties of Kronecker products, the authors derive cheaper and more accurate estimates than existing methods, with bounds that are always sharp and require only a few singular values—making them ideal for large-scale TLS problems.

ABSTRACT

This paper concerns cheaply computable formulas and bounds for the condition number of the TLS problem. For a TLS problem with data $A$, $b$, two formulas are derived that are simpler and more compact than the known results in the literature. One is derived by exploiting the properties of Kronecker products of matrices. The other is obtained by making use of the singular value decomposition (SVD) of $[A \,\,b]$, which allows us to compute the condition number cheaply and accurately. We present lower and upper bounds for the condition number that involve the singular values of $[A \,\, b]$ and the last entries of the right singular vectors of $[A \,\, b]$. We prove that they are always sharp and can estimate the condition number accurately by no more than four times. Furthermore, we establish a few other lower and upper bounds that involve only a few singular values of $A$ and $[A \,\, b]$. We discuss how tight the bounds are. These bounds are particularly useful for large scale TLS problems since for them any formulas and bounds for the condition number involving all the singular values of $A$ and/or $[A \ b]$ are too costly to be computed. Numerical experiments illustrate that our bounds are sharper than a known approximate condition number in the literature.

Motivation & Objective

• To develop computationally efficient and accurate formulas for the TLS condition number, overcoming the high cost of existing methods involving all singular values.
• To derive lower and upper bounds for the TLS condition number that depend only on a few key singular values and right singular vector components of [A b], enabling scalability to large problems.
• To improve upon known approximate condition numbers in the literature by providing tighter, more reliable bounds that are always sharp within a factor of four.
• To demonstrate numerically that the proposed bounds significantly outperform existing approximations, especially when the smallest singular values of A and [A b] are close.

Proposed method

• Derive a new closed-form expression for the TLS condition number using properties of Kronecker products, simplifying computation and improving numerical stability.
• Develop an alternative formula based on the SVD of the augmented matrix [A b], enabling cheaper and more accurate computation of the condition number.
• Establish sharp lower and upper bounds involving the singular values of [A b] and the last entries of its right singular vectors, proving they are always within a factor of four of the true condition number.
• Propose additional bounds using only a small number of leading singular values of A and [A b], suitable for large-scale problems where full SVD is infeasible.
• Utilize iterative SVD solvers to compute only the required singular values, making the bounds practical for large-scale TLS problems.
• Validate the bounds through numerical experiments comparing them with the known approximate condition number from [Björck et al., 2000], using synthetic test problems with controlled singular value structures.

Experimental results

Research questions

• RQ1 Can we derive a simpler and more computationally efficient formula for the TLS condition number than existing results?
• RQ2 How can we construct sharp, cheap-to-compute bounds for the TLS condition number that depend only on a few singular values?
• RQ3 Are the proposed bounds always sharp, and can they estimate the true condition number within a bounded factor (e.g., four times) across all test cases?
• RQ4 How do the new bounds compare in tightness and accuracy to the approximate condition number in [Björck et al., 2000], especially in ill-conditioned scenarios?
• RQ5 Can the proposed bounds be effectively used in large-scale TLS problems where full SVD computation is prohibitive?

Key findings

• The proposed SVD-based formula for the TLS condition number enables cheaper and more accurate computation than the method in [Björck et al., 2000].
• The new bounds involving the last entries of the right singular vectors of [A b] are always sharp and estimate the true condition number within a factor of four.
• For cases where σₙ and σₙ₊₁ are close (e.g., α ≤ 10⁻⁵), the proposed bound κ̄₁ʳ² significantly outperforms the approximate condition number in [Björck et al., 2000], improving by several orders of magnitude.
• The bounds that depend only on a few leading singular values of A and [A b] are both computationally feasible and highly accurate for large-scale problems.
• Numerical experiments confirm that the new bounds are consistently tighter than the known approximate condition number, especially in ill-conditioned regimes.
• The construction of test problems ensures controlled singular value structures, allowing clear validation of bound sharpness and accuracy across varying conditionings.