[Paper Review] Simultaneous Inference of Covariances

Xiao Han, Wei Biao Wu|arXiv (Cornell University)|Sep 2, 2011

Statistical Methods and Bayesian Inference3 citations

TL;DR

This paper establishes the asymptotic Gumbel distribution of the maximum deviation in high-dimensional sample covariance matrices under mild dependence conditions on data entries, generalizing prior i.i.d. assumptions. It provides a theoretical foundation for simultaneous inference on covariance structures in high-dimensional settings, enabling tests for bandedness, stationarity, and identity covariance matrices.

ABSTRACT

We consider asymptotic distributions of maximum deviations of sample covariance matrices, a fundamental problem in high-dimensional inference of covariances. Under mild dependence conditions on the entries of the data matrices, we establish the Gumbel convergence of the maximum deviations. Our result substantially generalizes earlier ones where the entries are assumed to be independent and identically distributed, and it provides a theoretical foundation for high-dimensional simultaneous inference of covariances.

Motivation & Objective

• To develop a theoretical framework for simultaneous inference of high-dimensional covariance matrices beyond i.i.d. assumptions.
• To generalize existing results on maximal deviation of sample covariances by allowing weak dependence among entries.
• To provide a rigorous asymptotic distributional theory for the maximum deviation $ M_n = \max_{i<j} |\hat{\sigma}_{ij} - \sigma_{ij}| $ under general dependence.
• To support practical inference on covariance structure, such as testing for bandedness, stationarity, or identity covariance.
• To extend the applicability of extreme value theory to high-dimensional covariance estimation under weak dependence.

Proposed method

• Derives the asymptotic distribution of the maximum deviation $ M_n $ of sample covariance entries from their population counterparts.
• Introduces weak dependence conditions on the data vector $ (X_{11}, \dots, X_{1m}) $, including moment and dependence strength constraints via $ \gamma(n, b_n) $ and $ r_{n,i,j} $.
• Uses a self-normalized version of $ M_n $ to achieve convergence to the Gumbel distribution under mild dependence.
• Employs Poisson approximation via the moment method to analyze rare events in high-dimensional covariance deviations.
• Applies Gaussian approximation results (Zaĭtsev, 1987) to control tail behavior under weak dependence.
• Establishes conditions under which the limiting distribution of $ M_n $ converges to the Gumbel distribution, even when entries are not i.i.d.

Experimental results

Research questions

• RQ1 Under what dependence conditions on high-dimensional data entries does the maximum deviation of sample covariances converge to the Gumbel distribution?
• RQ2 Can the theoretical framework for maximal covariance deviation be extended beyond i.i.d. entries to weakly dependent processes?
• RQ3 How do moment and dependence strength conditions affect the asymptotic behavior of extreme sample covariance deviations?
• RQ4 What are the implications of this limit theory for testing high-dimensional covariance structures such as bandedness or stationarity?
• RQ5 Is the Gumbel limit distribution robust to weak dependence, and what are the necessary and sufficient conditions for this convergence?

Key findings

• The maximum deviation $ M_n = \max_{i<j} |\hat{\sigma}_{ij} - \sigma_{ij}| $ converges in distribution to the Gumbel distribution under mild dependence conditions.
• The result generalizes Jiang (2004) and Cai and Jiang (2011) by allowing weak dependence among data entries, not just i.i.d. or $ s_n $-dependent cases.
• The conditions $ \gamma(n, b_n) = o(1) $ and $ \gamma(n, b_n) \log b_n = o(1) $ are shown to be sufficient for Gumbel convergence, with equivalent formulations via the function $ G_n(t) $.
• The limiting distribution is robust to weak dependence, as demonstrated through examples of linear and nonlinear processes satisfying the technical conditions.
• The theory supports practical tests for covariance structure, such as testing $ H_0: \Sigma_n = I_m $, bandedness, or stationarity, via the extreme value behavior of $ M_n $.
• The Poisson approximation method is validated with $ Q_{n,d} \to \lambda^d / d! $, confirming the convergence of the number of large deviations to a Poisson limit.