[Paper Review] RSVP-graphs: Fast High-dimensional Covariance Matrix Estimation under Latent Confounding

Rajen D. Shah, Benjamin Frot+2 more|arXiv (Cornell University)|Nov 2, 2018

Statistical Methods and Inference30 references4 citations

TL;DR

This paper proposes RSVP-graphs, a fast and scalable method for estimating high-dimensional covariance matrices under latent confounding by projecting observed data onto the right singular vectors of the data matrix. The method recovers the true covariance up to an unknown scale factor, enabling robust estimation of correlation matrices and network structures even when the gap between latent factor and idiosyncratic noise eigenvalues is small.

ABSTRACT

In this work we consider the problem of estimating a high-dimensional $p imes p$ covariance matrix $Σ$, given $n$ observations of confounded data with covariance $Σ+ ΓΓ^T$, where $Γ$ is an unknown $p imes q$ matrix of latent factor loadings. We propose a simple and scalable estimator based on the projection on to the right singular vectors of the observed data matrix, which we call RSVP. Our theoretical analysis of this method reveals that in contrast to PCA-based approaches, RSVP is able to cope well with settings where the smallest eigenvalue of $Γ^T Γ$ is close to the largest eigenvalue of $Σ$, as well as settings where the eigenvalues of $Γ^T Γ$ are diverging fast. It is also able to handle data that may have heavy tails and only requires that the data has an elliptical distribution. RSVP does not require knowledge or estimation of the number of latent factors $q$, but only recovers $Σ$ up to an unknown positive scale factor. We argue this suffices in many applications, for example if an estimate of the correlation matrix is desired. We also show that by using subsampling, we can further improve the performance of the method. We demonstrate the favourable performance of RSVP through simulation experiments and an analysis of gene expression datasets collated by the GTEX consortium.

Motivation & Objective

• To address the challenge of estimating high-dimensional covariance matrices when data are confounded by unobserved latent factors.
• To develop a computationally efficient method that remains effective even when the eigenvalues of the latent factor loading matrix are not well-separated from those of the true covariance matrix.
• To estimate the true covariance matrix Σ up to an unknown positive scale factor, which suffices for applications like correlation matrix estimation or network discovery.
• To avoid the need for estimating the number of latent factors q, which is often unknown and difficult to identify in practice.
• To improve performance via subsampling while maintaining theoretical consistency and robustness under elliptical distributions.

Proposed method

• The method uses the right singular vectors V of the column-centered data matrix X to construct the estimator Σ̂_rsvp = VV^T, termed the Right Singular Vector Projection (RSVP).
• The approach is inherently scale-free, recovering Σ only up to an unknown positive scalar factor, which is sufficient for correlation matrix estimation and network structure learning.
• The method does not require knowledge or estimation of the number of latent factors q, making it robust in high-dimensional settings with weak or overlapping eigenvalue separation.
• Subsampling is applied to improve finite-sample performance and enhance stability of the estimator.
• Theoretical analysis shows that RSVP maintains consistency under a broad range of conditions, including when the smallest eigenvalue of Γ^TΓ is close to the largest eigenvalue of Σ.
• The method is distributionally robust: if data rows have elliptical distributions, the estimator's distribution matches that under Gaussianity, as shown in Proposition 4.

Experimental results

Research questions

• RQ1 Can a fast, scalable method estimate high-dimensional covariance matrices under latent confounding without requiring knowledge of the number of latent factors?
• RQ2 How does the method perform when the eigenvalue gap between latent factors and idiosyncratic noise is small or non-existent?
• RQ3 To what extent does the scale-free nature of the estimator affect downstream inference, such as correlation matrix estimation or network discovery?
• RQ4 Can subsampling improve the finite-sample performance of the estimator while preserving theoretical consistency?
• RQ5 Does the method maintain robustness under non-Gaussian elliptical distributions?

Key findings

• RSVP achieves consistent estimation of the true covariance matrix Σ up to an unknown scale factor, even when the smallest eigenvalue of Γ^TΓ is close to the largest eigenvalue of Σ.
• The method remains effective in settings where traditional principal component removal fails due to lack of eigenvalue separation or instability in top PC estimation.
• Theoretical analysis confirms that RSVP maintains consistency under a broad class of high-dimensional models, including when eigenvalues of Γ^TΓ diverge rapidly.
• Subsampling significantly improves the finite-sample performance of the estimator, particularly in moderate to high-dimensional settings.
• The estimator is distributionally robust: under elliptical distributions, its sampling distribution matches that under Gaussianity, as proven in Proposition 4.
• Empirical validation on GTEX gene expression data shows RSVP outperforms existing methods in enriching biologically relevant pathways and constructing accurate gene networks.