[Paper Review] Error Bound for Compound Wishart Matrices

Ilya Soloveychik|arXiv (Cornell University)|Feb 23, 2014

Random Matrices and Applications21 references3 citations

TL;DR

This paper establishes non-asymptotic error bounds for compound Wishart matrices of the form $ W = \frac{1}{n}XBX^T $, where $ X $ has i.i.d. standard Gaussian entries and $ B $ is an arbitrary $ n \times n $ matrix. Using concentration inequalities and Gaussian measure techniques, it derives a bound on the expected spectral norm of the deviation $ \|W - W^0\| $, showing that the sample complexity scales as $ \mathcal{O}(\sqrt{p} \ln^2 p) $ for accurate estimation of the mean matrix.

ABSTRACT

In this paper we consider non-asymptotic behavior of the real compound Wishart matrices that generalize the classical real Wishart distribution. In particular, we consider matrices of the form 1/nXBX', where X consists of real centered Gaussian elements and B is an arbitrary real matrix and sequences of such matrices for varying n. We show how the expectation of deviations from the mean can be bounded for compound Wishart matrices.

Motivation & Objective

• To analyze the non-asymptotic behavior of compound Wishart matrices $ W = \frac{1}{n}XBX^T $, where $ X $ has i.i.d. standard Gaussian entries and $ B $ is an arbitrary matrix.
• To derive finite-sample bounds on the expected spectral norm of the deviation $ \|W - \mathbb{E}[W]\| $, addressing the precision of estimating the mean Wishart matrix.
• To investigate the sample complexity required to estimate the expectation of compound Wishart matrices accurately, especially when $ p $ and $ n $ are of the same order.
• To extend results to sequences of Wishart matrices $ W_n = \frac{1}{n}X B_n X^T $ with $ \mathrm{Tr}(B_n) = n $, ensuring controlled spectral properties.

Proposed method

• Uses the representation $ W = \frac{1}{n} \Theta^{1/2} Y B X^T \Theta^{1/2} $, where $ Y $ has i.i.d. standard normal entries, to decouple the scale matrix $ \Theta $ from the structure of $ B $.
• Applies Bernstein-type concentration inequalities for matrix-valued random variables via the Stein method and Lieb's theorem.
• Employs a union bound over sparse vectors in $ \text{Reg}_p(r) $, the set of $ r $-sparse vectors with $ \pm 1 $ entries, to control the maximum quadratic form $ (W'x, y) $.
• Derives a tail bound on the spectral norm $ \|W'\| $ by controlling the deviation of quadratic forms over sparse supports using Gaussian tail estimates.
• Uses the fact that $ \mathbb{E}[\|W - W^0\|] \leq \frac{C \lceil \ln(2p) \rceil^2 \sqrt{p}}{n} $, where $ C $ depends on $ \|B\|_{\text{Frob}} $ and $ \|B\| $, to derive the final bound.
• Applies integration of tail bounds to convert tail probability inequalities into expectation bounds on the spectral norm.

Experimental results

Research questions

• RQ1 What is the non-asymptotic deviation of a compound Wishart matrix from its mean in terms of spectral norm?
• RQ2 How many samples $ n $ are required to estimate the expected compound Wishart matrix $ W^0 $ with high probability and controlled error?
• RQ3 What is the sample complexity for estimating the mean of a sequence of compound Wishart matrices $ W_n = \frac{1}{n}X B_n X^T $ with $ \mathrm{Tr}(B_n) = n $?
• RQ4 How does the error bound scale with dimension $ p $, especially when $ n \approx p $?
• RQ5 Can the concentration of measure technique yield tight bounds on the spectral norm of $ W - W^0 $ under general $ B $, including non-symmetric or structured matrices?

Key findings

• The expected spectral norm of the deviation $ \|W - W^0\| $ is bounded by $ \mathbb{E}[\|W - W^0\|] \leq \frac{24 \lceil \ln(2p) \rceil^2 \sqrt{p} (4\sigma + \kappa \sqrt{\pi})}{n} $, where $ \sigma $ and $ \kappa $ depend on $ \|B\|_{\text{Frob}} $ and $ \|B\| $, respectively.
• For sequences of matrices $ \{B_n\} $ with $ \mathrm{Tr}(B_n) = n $, the bound holds uniformly with $ \sigma_n \leq \sigma $, $ \kappa_n \leq \kappa $, ensuring stability across $ n $.
• The sample complexity required to estimate $ W^0 $ with high probability is $ \mathcal{O}(\sqrt{p} \ln^2 p) $, which is subquadratic in $ p $ and suitable for high-dimensional settings.
• The bound is derived via a union bound over $ \text{Reg}_p(r) \times \text{Reg}_p(s) $, the set of sparse vectors with $ r,s \leq p $, and Gaussian tail estimates.
• The result holds even when $ B $ is not symmetric, extending applicability to cases such as skew-symmetric $ B $, which arise in representation-theoretic settings.
• The final bound is robust to the choice of $ B $, provided its Frobenius and spectral norms are controlled, and applies to both fixed $ p $ and growing $ p(n) $.