[Paper Review] Spectral norm of products of random and deterministic matrices

Roman Vershynin|arXiv (Cornell University)|Dec 12, 2008

Random Matrices and Applications27 references4 citations

TL;DR

This paper establishes that the spectral norm of a product matrix $ W = BA $, where $ A $ is a random $ N \times n $ matrix with independent, mean-zero entries having finite $ (4+\varepsilon) $-th moments and $ B $ is a fixed $ m \times N $ matrix with $ \|B\| \leq 1 $, is bounded in expectation by $ C(\varepsilon)(\sqrt{m} + \sqrt{n}) $. The result shows that such structured random matrices behave like i.i.d. random matrices in terms of spectral norm, under minimal moment assumptions, and extends prior bounds on the smallest singular value of random matrices.

ABSTRACT

We study the spectral norm of matrices M that can be factored as M=BA, where A is a random matrix with independent mean zero entries, and B is a fixed matrix. Under the (4+epsilon)-th moment assumption on the entries of A, we show that the spectral norm of such an m by n matrix M is bounded by \sqrt{m} + \sqrt{n}, which is sharp. In other words, in regard to the spectral norm, products of random and deterministic matrices behave similarly to random matrices with independent entries. This result along with the previous work of M. Rudelson and the author implies that the smallest singular value of a random m times n matrix with i.i.d. mean zero entries and bounded (4+epsilon)-th moment is bounded below by \sqrt{m} - \sqrt{n-1} with high probability.

Motivation & Objective

• To extend sharp spectral norm bounds for i.i.d. random matrices to products of random and deterministic matrices.
• To analyze the spectral norm of $ W = BA $, where $ A $ has independent, mean-zero entries with $ (4+\varepsilon) $-th moment bounded by 1, and $ B $ is fixed.
• To establish that the spectral norm of such products is bounded by $ C(\varepsilon)(\sqrt{m} + \sqrt{n}) $, independent of the intermediate dimension $ N $.
• To complete the analysis of the smallest singular value of random rectangular matrices initiated by Rudelson and Vershynin.

Proposed method

• Uses a decomposition of the matrix $ B $ into large and small columns based on $ \ell^2 $-norms of its columns.
• Applies a conditional moment method: condition on the maximum entry of $ A $ being bounded, preserving independence and controlling moments.
• Employs a truncation argument via Markov's inequality to control tail probabilities of matrix entries.
• Applies Corollary 4.2 (a moment bound for matrices with bounded entries) conditionally on the event that $ \max_{i,j} |a_{ij}| \leq t $.
• Uses a net argument and symmetrization to bound the spectral norm under moment assumptions.
• Combines estimates for large-column and small-column submatrices using the triangle inequality on $ \|BA\| \leq \|B_I A_I\| + \|B_{I^c} A_{I^c}\| $.

Experimental results

Research questions

• RQ1 Can the spectral norm of $ W = BA $, where $ A $ has independent, mean-zero entries and $ B $ is deterministic, be bounded by $ C(\varepsilon)(\sqrt{m} + \sqrt{n}) $ under a $ (4+\varepsilon) $-th moment condition?
• RQ2 Does the spectral norm of such structured random matrices behave like that of i.i.d. random matrices, even when entries are dependent due to the factorization?
• RQ3 Can the bound be made independent of the intermediate dimension $ N $, the size of the random matrix $ A $?
• RQ4 How does this result extend the analysis of the smallest singular value of random matrices?

Key findings

• The expected spectral norm of $ W = BA $ satisfies $ \mathbb{E}\|W\| \leq C(\varepsilon)(\sqrt{m} + \sqrt{n}) $, where $ C(\varepsilon) $ depends only on $ \varepsilon $.
• The bound is sharp and independent of the intermediate dimension $ N $, even when $ N \gg m,n $.
• The result holds under the minimal $ (4+\varepsilon) $-th moment assumption, which is known to be necessary for such bounds.
• The proof yields a stronger estimate: $ \mathbb{E}\|W\| \leq C(\varepsilon)(\|B\|\sqrt{n} + \|B\|_{\rm{HS}}) $, valid for arbitrary $ B $.
• The result implies that the smallest singular value of an $ m \times n $ random matrix with i.i.d. entries and bounded $ (4+\varepsilon) $-th moment is bounded below by $ \sqrt{m} - \sqrt{n-1} $ with high probability.
• The analysis applies to sample covariance matrices of the form $ \Sigma = \frac{1}{n}WW^* $, where $ W = BA $, and shows that the largest eigenvalue of $ \Sigma $ is bounded by $ C_1(\varepsilon)(1 + m/n) $.