[Paper Review] randUTV: A blocked randomized algorithm for computing a rank-revealing UTV factorization

Per‐Gunnar Martinsson, Gregorio Quintana‐Ortí+1 more|arXiv (Cornell University)|Mar 3, 2017

Stochastic Gradient Optimization Techniques20 references4 citations

TL;DR

This paper introduces randUTV, a randomized blocked algorithm for computing a rank-revealing UTV factorization that achieves SVD-like accuracy for low-rank approximation while being significantly faster and more easily parallelizable than traditional SVD. The method uses randomized subspace iteration and incremental, non-iterative block processing to compute orthonormal U and V matrices and a triangular T matrix, enabling early termination and high performance on modern architectures, with diagonal entries of T providing accurate singular value approximations.

ABSTRACT

This manuscript describes the randomized algorithm randUTV for computing a so called UTV factorization efficiently. Given a matrix $A$, the algorithm computes a factorization $A = UTV^{*}$, where $U$ and $V$ have orthonormal columns, and $T$ is triangular (either upper or lower, whichever is preferred). The algorithm randUTV is developed primarily to be a fast and easily parallelized alternative to algorithms for computing the Singular Value Decomposition (SVD). randUTV provides accuracy very close to that of the SVD for problems such as low-rank approximation, solving ill-conditioned linear systems, determining bases for various subspaces associated with the matrix, etc. Moreover, randUTV produces highly accurate approximations to the singular values of $A$. Unlike the SVD, the randomized algorithm proposed builds a UTV factorization in an incremental, single-stage, and non-iterative way, making it possible to halt the factorization process once a specified tolerance has been met. Numerical experiments comparing the accuracy and speed of randUTV to the SVD are presented. These experiments demonstrate that in comparison to column pivoted QR, which is another factorization that is often used as a relatively economic alternative to the SVD, randUTV compares favorably in terms of speed while providing far higher accuracy.

Motivation & Objective

• To develop a fast, scalable, and accurate alternative to the SVD for low-rank matrix approximation.
• To enable incremental, non-iterative computation of a rank-revealing UTV factorization with early termination capability.
• To achieve high performance on multicore CPUs, GPUs, and distributed systems through blocked matrix operations and randomized sampling.
• To provide accurate approximations to the singular values of a matrix via the diagonal of the triangular factor T.
• To outperform column pivoted QR in accuracy while maintaining competitive speed, especially for large matrices.

Proposed method

• The algorithm uses randomized subspace iteration to build orthonormal bases for the dominant left and right singular subspaces of the input matrix A.
• It processes the matrix in blocks of size b, applying unitary transformations to reduce each block to upper triangular form in a blocked, single-stage process.
• The method computes a UTV factorization A = U T V* where U and V are unitary, and T is upper triangular, with T's diagonal entries approximating the singular values of A.
• The algorithm incorporates power iterations (q ≥ 0) to enhance accuracy, with q=1 or 2 yielding near-optimal low-rank approximations.
• It is designed to be non-iterative and incremental, allowing early termination once a desired rank or tolerance is reached.
• The implementation leverages high-performance matrix-multiplication kernels (e.g., BLAS3) for efficiency on modern architectures.

Experimental results

Research questions

• RQ1 Can a randomized blocked algorithm achieve SVD-like accuracy in low-rank approximation while being significantly faster than standard SVD?
• RQ2 Does the randUTV algorithm provide more accurate singular value approximations than column pivoted QR or QLP factorization?
• RQ3 How does the performance of randUTV scale with matrix size and number of processors compared to SVD and CPQR?
• RQ4 To what extent does over-sampling improve the accuracy of randUTV, and is the cost justified?
• RQ5 Can the algorithm be efficiently parallelized and optimized for modern multicore and distributed-memory systems?

Key findings

• randUTV achieves low-rank approximation errors that are very close to the theoretical minimum, significantly outperforming column pivoted QR in accuracy.
• With q=1 or q=2 power iterations, randUTV's approximation errors are nearly optimal across all test matrices, including those with slow decay and singular value gaps.
• The diagonal entries of the triangular factor T in randUTV provide highly accurate approximations to the singular values of A, outperforming both CPQR and QLP factorizations.
• For large matrices, randUTV is substantially faster than SVD (e.g., Intel MKL dgesvd), and its relative speed advantage increases with the number of processors.
• The algorithm’s performance is highly scalable on modern architectures due to its use of compute-intensive blocked operations and efficient memory access patterns.
• Over-sampling provides only marginal accuracy gains, making it generally unnecessary for most applications.