[Paper Review] The Convergence of Least-Squares Progressive Iterative Approximation with Singular Iterative Matrix

Hongwei Lin, Qi Cao+1 more|arXiv (Cornell University)|Jul 28, 2017

Advanced Numerical Analysis Techniques13 references3 citations

TL;DR

This paper proves that Least-Squares Progressive Iterative Approximation (LSPIA) converges even when the iterative matrix is singular, extending its applicability to large-scale geometric fitting problems with rank-deficient systems. The method converges to the minimum norm solution via the Moore-Penrose pseudo-inverse, validated through theoretical analysis and a real-world trivariate B-spline fitting example with 43,994 data points and a 20,790×20,790 singular system.

ABSTRACT

Developed in [Deng and Lin, 2014], Least-Squares Progressive Iterative Approximation (LSPIA) is an efficient iterative method for solving B-spline curve and surface least-squares fitting systems. In [Deng and Lin 2014], it was shown that LSPIA is convergent when the iterative matrix is nonsingular. In this paper, we will show that LSPIA is still convergent even the iterative matrix is singular.

Motivation & Objective

• To establish convergence of LSPIA when the coefficient matrix in the least-squares fitting system is singular.
• To extend the theoretical foundation of LSPIA beyond the previously known nonsingular case.
• To support practical applications involving large-scale or incomplete data, such as tetrahedral mesh fitting and image reconstruction with holes.
• To demonstrate that LSPIA converges to the minimum Euclidean norm solution (Moore-Penrose pseudo-inverse) under specific conditions.

Proposed method

• Reformulates LSPIA as an iterative process involving vector distribution and vector gathering across control points.
• Defines the difference vector for data points (DVD) and the difference vector for control points (DVC) using weighted averaging.
• Analyzes convergence using eigen-decomposition and spectral radius conditions on the matrix $\alpha A^T A$.
• Derives the iterative format $P^{(k+1)} = (I - \alpha A^T A)P^{(k)} + \alpha A^T Q$ for convergence analysis.
• Applies the Moore-Penrose pseudo-inverse to characterize the limit solution when $A^T A$ is singular.
• Uses orthogonal decomposition and spectral properties of symmetric positive semidefinite matrices to prove convergence to the minimum norm solution.

Experimental results

Research questions

• RQ1 Does LSPIA converge when the iterative matrix $A^T A$ is singular, despite the absence of a unique solution?
• RQ2 Can LSPIA still converge to a stable and meaningful solution in the presence of rank-deficient systems arising in geometric modeling?
• RQ3 What is the limiting solution of LSPIA when $A^T A$ is singular, and does it correspond to the minimum norm solution?
• RQ4 Under what conditions does LSPIA converge to the Moore-Penrose pseudo-inverse solution of the least-squares system?

Key findings

• LSPIA converges to a stable solution even when the coefficient matrix $A^T A$ is singular, as proven via spectral analysis of the iterative matrix.
• When the diagonal elements of the matrix $\Lambda$ are equal, LSPIA converges to the Moore-Penrose pseudo-inverse solution of the least-squares system.
• The limit solution is the minimum Euclidean norm solution when the initial control points $P^{(0)} = 0$, regardless of the rank of $A^T A$.
• In a real-world example with 43,994 tetrahedral mesh vertices, LSPIA converged to a solution with a least-squares fitting error of $9.97 \times 10^{-5}$ in 5.50 seconds, despite the $20,790 \times 20,790$ matrix having rank 12,628.
• The convergence is guaranteed as long as the spectral radius $\rho(\alpha A^T A) \leq 1$, ensuring stability of the iterative process.