[Paper Review] On the two mutually independent factors that determine the convergence of least-squares projection method

Shukai Du, Nailin Du|arXiv (Cornell University)|Jun 3, 2014

Numerical methods in inverse problems5 references3 citations

TL;DR

This paper identifies two mutually independent geometric factors—kernel approximability and offset angle—that determine the convergence of the least-squares projection method for bounded linear operators in Hilbert spaces. It establishes that strong convergence occurs if and only if kernel approximability holds and the offset angle remains uniformly bounded below $\pi/2$, providing a geometric framework to assess and design convergent approximation schemes.

ABSTRACT

This paper investigates the least-squares projection method for bounded linear operators, which provides a natural regularization scheme by projection for many ill-posed problems. Yet, without additional assumptions, the convergence of this approximation scheme cannot be guaranteed. We reveal that the convergence of least-squares projection method is determined by two independent factors -- the kernel approximability and the offset angle. The kernel approximability is a necessary condition of convergence described with kernel $N(T)$ and its subspaces $N(T){\cap}X_n$, and we give several equivalent characterizations for it (Theorem 1). The offset angle of $X_n$ is defined as the largest canonical angle between space $T^*T(X_n)$ and $T^{\dagger}T(X_n)$ (which are subspaces of $N(T)^\bot$), and it geometrically reflects the rate of convergence (Theorem 2). The paper also presents new observations for the unconvergence examples of Seidman [10, Example 3.1] and Du [2, Example 2.10] under the notions of kernel approximability and offset angle.

Motivation & Objective

• To identify the necessary and sufficient conditions for strong convergence of the least-squares projection method when approximating the Moore-Penrose inverse of a bounded linear operator.
• To provide geometric insight into convergence by introducing the offset angle and kernel approximability as independent, interpretable factors.
• To resolve the lack of geometric intuition in existing convergence criteria, such as uniform boundedness of the norms of the regularized inverses.
• To re-analyze known non-convergent examples (Seidman [10], Du [2]) using the new geometric concepts to clarify their failure mechanisms.

Proposed method

• Introduces the concept of kernel approximability as the strong convergence of orthogonal projections onto the kernels of the approximated operators, i.e., $\lim_{n\to\infty} P_{\mathcal{N}(T_n)} = P_{\mathcal{N}(T)}$.
• Defines the offset angle $\theta_n$ as the largest canonical angle between the subspaces $T^*T(X_n)$ and $T^\dagger T(X_n)$ within $\mathcal{N}(T)^\perp$.
• Establishes that strong convergence of $T_n^\dagger$ to $T^\dagger$ is equivalent to kernel approximability and $\sup_n \theta_n < \pi/2$.
• Provides multiple equivalent characterizations of kernel approximability, including bounded-weak convergence and convergence of projections onto the graphs of the adjoint operators.
• Uses the geometric structure of $\mathcal{N}(T)^\perp$ to analyze the interplay between the range of $T^*T$ and the range of $T^\dagger T$ on $X_n$.
• Derives error bounds of the form $\|T_n^\dagger y - T^\dagger y\| \leq \sqrt{1 + \tan^2 \theta_n} \cdot \mathrm{dist}(T^\dagger y, X_n)$, linking convergence rate to the offset angle.

Experimental results

Research questions

• RQ1 What are the necessary and sufficient geometric conditions for strong convergence of the least-squares projection method?
• RQ2 How can the convergence behavior be decomposed into independent, interpretable factors?
• RQ3 Why do some standard approximation schemes fail to converge, and what geometric property explains their failure?
• RQ4 Can the offset angle be used to predict or control the rate of convergence?
• RQ5 How do kernel approximability and offset angle relate to classical convergence criteria like uniform boundedness of $\|T_n^\dagger T\|$?

Key findings

• Strong convergence of the least-squares projection method is equivalent to the joint satisfaction of kernel approximability and $\sup_n \theta_n < \pi/2$, where $\theta_n$ is the offset angle.
• Kernel approximability is equivalent to $\lim_{n\to\infty} P_{\mathcal{N}(T_n)} = P_{\mathcal{N}(T)}$, and also to bounded-weak convergence of the projections onto the graphs of the adjoint operators.
• When $\mathcal{N}(T) = \{0\}$, strong convergence holds if and only if $\sup_n \theta_n < \pi/2$, and the error bound is $\|T_n^\dagger y - T^\dagger y\| \leq \sqrt{1 + \tan^2 \theta_n} \cdot \mathrm{dist}(T^\dagger y, X_n)$.
• For finite-dimensional $\mathcal{N}(T)$ and closed range, kernel approximability is equivalent to strong convergence, and the same error bound holds for sufficiently large $n$.
• The offset angle $\theta_n$ geometrically measures the angle between $T^*T(X_n)$ and $T^\dagger T(X_n)$ in $\mathcal{N}(T)^\perp$, and smaller angles imply faster convergence.
• If $T$ satisfies a coercivity condition $|\langle Tu,u\rangle| \geq \alpha \|u\|^2$ and $|\langle Tu,v\rangle| \leq \beta \|u\|\|v\|$, then $\sup_n \sqrt{1 + \tan^2 \theta_n} \leq \|T\| \|T^{-1}\| \leq \beta / \alpha$.