[Paper Review] The PowerURV algorithm for computing rank-revealing full factorizations

Abinand Gopal, Per‐Gunnar Martinsson|arXiv (Cornell University)|Dec 14, 2018

Stochastic Gradient Optimization Techniques10 references4 citations

TL;DR

The PowerURV algorithm is a randomized, computationally efficient method for computing rank-revealing full factorizations of large matrices, leveraging randomized subspace iteration and level-3 BLAS operations. It achieves near-optimal low-rank approximations with performance comparable to SVD but at a fraction of the cost, especially in regimes where the numerical rank is not significantly smaller than matrix dimensions.

ABSTRACT

Many applications in scientific computing and data science require the computation of a rank-revealing factorization of a large matrix. In many of these instances the classical algorithms for computing the singular value decomposition are prohibitively computationally expensive. The randomized singular value decomposition can often be helpful, but is not effective unless the numerical rank of the matrix is substantially smaller than the dimensions of the matrix. We introduce a new randomized algorithm for producing rank-revealing factorizations based on existing work by Demmel, Dumitriu and Holtz [Numerische Mathematik, 108(1), 2007] that excels in this regime. The method is exceptionally easy to implement, and results in close-to optimal low-rank approximations to a given matrix. The vast majority of floating point operations are executed in level-3 BLAS, which leads to high computational speeds. The performance of the method is illustrated via several numerical experiments that directly compare it to alternative techniques such as the column pivoted QR factorization, or the QLP method by Stewart.

Motivation & Objective

• Address the computational inefficiency of full SVD for large matrices with moderate numerical rank.
• Overcome limitations of randomized SVD (RSVD) and column pivoted QR (CPQR) in regimes where target rank is not much smaller than matrix dimensions.
• Develop a method that combines the accuracy of SVD with the speed of randomized algorithms and CPQR.
• Provide a rank-revealing factorization that enables high-quality low-rank approximations through a single, efficient factorization.
• Ensure compatibility with modern high-performance architectures by relying on level-3 BLAS operations for optimal memory and communication efficiency.

Proposed method

• Use randomized subspace iteration with power iterations to enhance the range of the random projection matrix.
• Apply an unpivoted QR factorization to the projected matrix to extract a basis for the column space.
• Construct a URV factorization where A = U R V*, with U having orthonormal columns, R upper triangular, and V orthogonal.
• Leverage the equivalence between PowerURV and the randomized SVD (RSVD) to inherit theoretical convergence guarantees.
• Use only matrix-matrix multiplications and unpivoted QR factorizations, enabling high performance on modern hardware via level-3 BLAS kernels.
• Ensure rank-revealing properties by aligning the leading columns of U with the dominant singular vectors through power iteration.

Experimental results

Research questions

• RQ1 Can a randomized algorithm achieve near-optimal low-rank approximations with computational cost significantly below that of full SVD?
• RQ2 How does the PowerURV algorithm compare in accuracy and speed to CPQR, full SVD, and the QLP method for rank-revealing factorizations?
• RQ3 To what extent does power iteration improve the quality of the randomized basis in the context of rank-revealing factorizations?
• RQ4 Can a rank-revealing factorization be computed efficiently using only level-3 BLAS operations while maintaining high numerical accuracy?
• RQ5 What is the relationship between PowerURV and the randomized SVD (RSVD) in terms of approximation quality and computational cost?

Key findings

• PowerURV achieves near-optimal low-rank approximations with computational cost substantially lower than full SVD, especially when the numerical rank is not much smaller than the matrix dimensions.
• The method matches the accuracy of the randomized SVD (RSVD) in practice, though RSVD slightly outperforms PowerURV due to one additional matrix-vector application.
• Numerical experiments show that PowerURV outperforms CPQR in terms of rank-revealing quality, avoiding the arbitrarily poor approximations known to occur with CPQR.
• The algorithm is highly efficient on modern hardware due to its reliance on level-3 BLAS operations, enabling high computational throughput.
• The PowerURV factorization enables high-quality low-rank approximations through a single, compact factorization that can be truncated at any rank.
• Theoretical equivalence between PowerURV and RSVD ensures that the method inherits strong convergence properties, particularly when power iteration is used.