[Paper Review] A geometric note on subspace updates and orthogonal matrix decompositions under rank-one modifications

Ralf Zimmermann|arXiv (Cornell University)|Nov 22, 2017

Matrix Theory and Algorithms18 references4 citations

TL;DR

This paper presents a geometric framework for efficiently updating orthogonal matrix factorizations—such as SVD or QR—under rank-one modifications to the original matrix. By modeling the subspace update as a geodesic path on the Grassmann manifold, it derives closed-form, O(np)-time updates for both the orthogonal factor $ U_{\text{new}} $ and the weight matrix $ W_{\text{new}} $, enabling exact and stable subspace tracking with minimal computational overhead.

ABSTRACT

In this work, we consider rank-one adaptations $X_{new} = X+ab^T$ of a given matrix $X\in \mathbb{R}^{n imes p}$ with known matrix factorization $X = UW$, where $U\in\mathbb{R}^{n imes p}$ is column-orthogonal, i.e. $U^TU=I$. Arguably the most important methods that produce such factorizations are the singular value decomposition (SVD), where $X=UW=UΣV^T$, and the QR-decomposition, where $X = UW = QR$. An elementary approach to produce a column-orthogonal matrix $U_{new}$, whose columns span the same subspace as the columns of the rank-one modified $X_{new} = X +ab^T$ is via applying a suitable coordinate change such that in the new coordinates, the update affects a single column and subsequently performing a Gram-Schmidt step for reorthogonalization. This may be interpreted as a rank-one adaptation of the $U$-factor in the SVD or a rank-one adaptation of the $Q$-factor in the QR-decomposition, respectively, and leads to a decomposition for the adapted matrix $X_{new} = U_{new}W_{new}$. By using a geometric approach, we show that this operation is equivalent to traveling from the subspace $\mathcal{S}= ext{ran}(X)$ to the subspace $\mathcal{S}_{new} = ext{ran}(X_{new})$ on a geodesic line on the Grassmann manifold and we derive a closed-form expression for this geodesic. In addition, this allows us to determine the subspace distance between the subspaces $\mathcal{S}$ and $\mathcal{S}_{new}$ without additional computational effort. Both $U_{new}$ and $W_{new}$ are obtained via elementary rank-one matrix updates in $\mathcal{O}(np)$ time for $n\gg p$.

Motivation & Objective

• To develop a numerically stable, efficient method for updating orthogonal matrix factorizations (e.g., SVD, QR) when the original matrix undergoes a rank-one modification.
• To establish a geometric interpretation of subspace updates as geodesic paths on the Grassmann manifold, enabling precise subspace tracking.
• To derive closed-form expressions for the updated orthogonal factor $ U_{\text{new}} $ and weight matrix $ W_{\text{new}} $, ensuring orthonormality and subspace invariance.
• To enable direct computation of the subspace distance between $ \operatorname{ran}(X) $ and $ \operatorname{ran}(X_{\text{new}}) $ without additional cost.
• To support applications in adaptive model reduction, computer vision, and signal processing by providing a scalable, numerically robust update scheme.

Proposed method

• Model the rank-one update $ X_{\text{new}} = X + ab^T $ as a geodesic motion on the Grassmann manifold, parameterized by a tangent vector $ \Delta $ of rank one.
• Use a coordinate transformation to reduce the update to a single-column reorthogonalization step, leveraging Gram-Schmidt-like reorthogonalization in the new basis.
• Derive $ U_{\text{new}} = U + (\alpha U w + \beta q) w^T $ and $ W_{\text{new}} = W + (U^T a + \gamma w) b^T $, where $ q $ is the normalized component of $ a $ orthogonal to $ \operatorname{ran}(X) $, and $ w $ is a normalized direction in the row space.
• Compute the subspace distance via $ \operatorname{Sdist} = \arccos\left( \frac{|\omega|}{\|g\|} \right) $, where $ \omega $ and $ g $ are derived from the update geometry.
• Implement the update using standard BLAS Level-2 operations (e.g., DGER), ensuring efficiency and compatibility with high-performance linear algebra libraries.
• Handle degenerate cases (e.g., $ a \in \operatorname{ran}(X) $) by detecting subspace invariance and returning the original factorization with adjusted $ W_{\text{new}} $.

Experimental results

Research questions

• RQ1 How can a rank-one update to a matrix with known orthogonal factorization be interpreted geometrically on the Grassmann manifold?
• RQ2 What is the closed-form expression for the updated orthogonal factor $ U_{\text{new}} $ that preserves orthonormality and spans the new column space?
• RQ3 Can the subspace distance between the original and updated subspaces be computed efficiently without additional matrix factorizations?
• RQ4 How can the update be implemented using only elementary rank-one operations to maintain computational efficiency?
• RQ5 What is the relationship between the tangent vector on the Grassmann manifold and the structure of the rank-one modification $ ab^T $?

Key findings

• The update of the orthogonal factor $ U $ and the weight matrix $ W $ can be computed in $ \mathcal{O}(np) $ time for $ n \gg p $, making it highly efficient for large-scale problems.
• The updated matrix $ X_{\text{new}} = U_{\text{new}} W_{\text{new}} $ exactly preserves the column space of $ X_{\text{new}} $, with $ U_{\text{new}} $ having orthonormal columns.
• The subspace distance between $ \operatorname{ran}(X) $ and $ \operatorname{ran}(X_{\text{new}}) $ is given by $ \operatorname{Sdist} = \arccos\left( \frac{|\omega|}{\|g\|} \right) $, computed directly from the update parameters.
• The method generalizes to both SVD and QR decompositions by treating $ U $ and $ Q $ uniformly as column-orthogonal matrices spanning the same subspace.
• The update formulas are numerically stable and compatible with standard BLAS routines, enabling integration into high-performance numerical pipelines.
• In the case $ a \in \operatorname{ran}(X) $, the method correctly detects subspace invariance and returns $ U_{\text{new}} = U $, with $ W_{\text{new}} $ updated only in the weight matrix.