[Paper Review] A Generalized Randomized Rank-Revealing Factorization

Grey Ballard, James Demmel+2 more|arXiv (Cornell University)|Sep 14, 2019

Sparse and Compressive Sensing Techniques17 references4 citations

TL;DR

This paper introduces a generalized randomized rank-revealing QR factorization (GURV) that enables efficient, stable, and communication-optimal computation of low-rank approximations for products of matrices and their inverses without explicitly forming them. The method achieves strong rank-revealing properties with high probability, proven theoretical bounds, and demonstrates tight empirical performance in numerical experiments.

ABSTRACT

We introduce a Generalized Randomized QR-decomposition that may be applied to arbitrary products of matrices and their inverses, without needing to explicitly compute the products or inverses. This factorization is a critical part of a communication-optimal spectral divide-and-conquer algorithm for the nonsymmetric eigenvalue problem. In this paper, we establish that this randomized QR-factorization satisfies the strong rank-revealing properties. We also formally prove its stability, making it suitable in applications. Finally, we present numerical experiments which demonstrate that our theoretical bounds capture the empirical behavior of the factorization.

Motivation & Objective

• To develop a rank-revealing factorization that works directly on products of matrices and their inverses without explicit computation.
• To ensure the factorization maintains strong rank-revealing properties in the Gu-Eisenstat sense with high probability.
• To achieve communication-optimality by relying only on QR and matrix multiplication, matching the complexity of matrix multiplication.
• To formally prove backward stability and theoretical probabilistic bounds for the factorization.
• To validate the tightness of theoretical bounds through extensive numerical experiments on structured singular value distributions.

Proposed method

• The method employs a randomized sampling strategy to construct a rank-revealing factorization $ A = URV $, where $ U $ and $ V $ are orthogonal, and $ R $ is upper triangular.
• It uses randomized range finders to extract a well-conditioned basis for the range of $ A $, ensuring $ \sigma_{\min}(R_{11}) \approx \sigma_r $ and $ \sigma_{\max}(R_{22}) \approx \sigma_{r+1} $.
• The algorithm is generalized to handle $ A = M_1 M_2^{-1} M_3 \cdots $ by applying the factorization to the product structure without computing the product explicitly.
• Theoretical analysis establishes that $ \|R_{11}^{-1} R_{12}\|_2 \leq \frac{4.04}{\delta} \sqrt{r(n-r)} + 1 $ with high probability.
• The method relies on communication-avoiding QR algorithms, ensuring arithmetic and communication complexity match that of matrix multiplication.
• Numerical experiments use matrices with stair-step and logarithmically spaced singular values to test bound tightness and empirical behavior.

Experimental results

Research questions

• RQ1 Can a randomized QR factorization be designed to maintain strong rank-revealing properties while avoiding explicit computation of matrix products or inverses?
• RQ2 What theoretical probabilistic bounds can be established for the condition number of $ R_{11} $, the norm of $ R_{22} $, and the off-diagonal block $ R_{12} $ in the factorization?
• RQ3 Does the proposed method achieve communication-optimality comparable to standard QR and matrix multiplication?
• RQ4 How tight are the theoretical bounds compared to empirical observations across different matrix structures and singular value gaps?
• RQ5 Can the factorization be applied effectively in the context of communication-optimal divide-and-conquer algorithms for the nonsymmetric eigenvalue problem?

Key findings

• Theoretical bounds for $ \sigma_r / \sigma_{\min}(R_{11}) $, $ \sigma_{\max}(R_{22}) / \sigma_{r+1} $, and $ \|R_{11}^{-1} R_{12}\|_2 $ are proven to hold with high probability, with the latter bounded by $ \frac{4.04}{\delta} \sqrt{r(n-r)} + 1 $.
• The bound $ \|R_{11}^{-1} R_{12}\|_2 \leq \frac{4.04}{\delta} \sqrt{r(n-r)} + 1 $ is shown to be tight, as empirical 97th percentiles closely follow the theoretical prediction.
• For matrices with a stair-step singular value distribution, deterministic bounds are looser than the probabilistic ones, but the latter remain tight and predictive.
• In experiments with $ n = 1500 $ and $ \sigma_r / \sigma_{r+1} = 10^7 $, the empirical 97th percentiles of the condition number and off-diagonal norms remain within the predicted theoretical bounds.
• The algorithm maintains strong rank-revealing properties even when applied to products of matrices and inverses, without explicit computation of the product.
• Numerical results confirm that the theoretical bounds are essentially tight, as the empirical distributions consistently lie just below the theoretical upper bounds across multiple test configurations.