[Paper Review] Notes on asymptotics of sample eigenstructure for spiked covariance models with non-Gaussian data

Iain M. Johnstone, Jeha Yang|arXiv (Cornell University)|Oct 24, 2018

Random Matrices and Applications16 references4 citations

TL;DR

This paper establishes asymptotic normality of sample eigenvalues and eigenvectors in spiked covariance models with non-Gaussian data, extending Paul (2007) to the non-Gaussian framework of Bai and Yao (2008). It provides a rigorous foundation for inference on leading sample eigenstructure under high-dimensional asymptotics, with key results on the limiting distribution of eigenvalues and projections of eigenvectors, accounting for non-Gaussian kurtosis via cumulant-based corrections.

ABSTRACT

These expository notes serve as a reference for an accompanying post Morales-Jimenez et al. [2018]. In the spiked covariance model, we develop results on asymptotic normality of sample leading eigenvalues and certain projections of the corresponding sample eigenvectors. The results parallel those of Paul [2007], but are given using the non-Gaussian model of Bai and Yao [2008]. The results are not new, and citations are given, but proofs are collected and organized as a point of departure for Morales-Jimenez et al. [2018].

Motivation & Objective

• To extend Paul (2007)’s results on asymptotic normality of sample eigenstructure to non-Gaussian data under the spiked covariance model.
• To provide a self-contained reference for the technical tools and proofs used in Morales-Jimenez et al. (2018) on sample correlation matrices.
• To analyze the impact of non-Gaussianity—specifically fourth-order cumulants—on the limiting distribution of sample eigenvalues and eigenvectors.
• To establish asymptotic normality of leading sample eigenvalues and projections of corresponding eigenvectors under the non-Gaussian model of Bai and Yao (2008).
• To support high-dimensional inference in PCA by deriving limiting distributions that account for non-Gaussian kurtosis and sample size scaling.

Proposed method

• Uses the non-Gaussian spiked covariance model of Bai and Yao (2008), where the population covariance matrix Σ has a few large eigenvalues (spikes) and the rest are bounded, with i.i.d. noise components.
• Applies the Marchenko-Pastur law to characterize the limiting spectral distribution of the noise component, with convergence of extreme eigenvalues to the bulk edge.
• Employs a perturbation lemma (Lemma 13) to derive asymptotic expansions for eigenvalues and eigenvectors under small random perturbations.
• Derives the limiting distribution of sample eigenvalues using the phase transition function ρ(ℓ,γ) = ℓ + γℓ/(ℓ−1), which determines the location of outliers above the bulk edge.
• Incorporates fourth-order cumulants of the data via Kνjj′ and tensor contractions to correct for non-Gaussian kurtosis in the asymptotic variance of eigenvalues and eigenvector projections.
• Uses the resolvent operator Hr(A) to decompose eigenvector perturbations and derives a quadratic error bound for the eigenvector deviation.

Experimental results

Research questions

• RQ1 How do non-Gaussian fourth-order cumulants affect the asymptotic distribution of sample eigenvalues in the spiked covariance model?
• RQ2 What is the limiting distribution of the leading sample eigenvalues when the population covariance has a few large eigenvalues and the data are non-Gaussian?
• RQ3 How do projections of sample eigenvectors onto population eigenvectors behave asymptotically under non-Gaussianity?
• RQ4 To what extent does the standardization of data (i.e., using correlation matrices) alter the asymptotic normality of PCA components compared to using covariance matrices?
• RQ5 Can the asymptotic normality of sample eigenstructure be established under the non-Gaussian model of Bai and Yao (2008), and how does it differ from the Gaussian case?

Key findings

• The sample eigenvalues corresponding to spikes ℓν > 1 + √γ converge almost surely to ρ(ℓν, γ) > bγ, where ρ(ℓ, γ) = ℓ + γℓ/(ℓ−1), establishing a phase transition in the outlier behavior.
• The limiting distribution of the leading sample eigenvalues is asymptotically normal, with a variance that depends on the population eigenvalue ℓν and the sample size ratio γ = p/n.
• The asymptotic variance of the sample eigenvalues includes a correction term proportional to the fourth-order cumulants of the data, specifically through the tensor Kνjj′ = ∑k,k′ pν,k pν,k′ κjj′kk′.
• Projections of sample eigenvectors onto the corresponding population eigenvectors are asymptotically normal, with a variance that depends on the cumulant tensor and the eigenvalue gap.
• The eigenvector perturbation bound (83) shows that the deviation of the sample eigenvector from the population eigenvector is asymptotically linear in the perturbation, with a quadratic error bound of order ‖B‖².
• The results confirm that non-Gaussianity—particularly excess kurtosis—induces a non-zero bias in the asymptotic distribution of eigenvalues and eigenvectors, which must be corrected for valid inference.