[Paper Review] EigenPrism: Inference for High-Dimensional Signal-to-Noise Ratios

Lucas Janson, Rina Foygel Barber+1 more|arXiv (Cornell University)|May 8, 2015

Statistical Methods and Inference28 references4 citations

TL;DR

EigenPrism is a novel, computationally efficient method for constructing valid confidence intervals for the $θ^2 = \|\bm{\Sigma}^{1/2}\bm{\beta}\|_2^2$ signal-to-noise ratio in high-dimensional linear models ($p > n$), without requiring sparsity assumptions or knowledge of the noise level. It achieves asymptotic validity under multivariate Gaussian design and finite-sample coverage, enabling inference on regression error, noise level, and genetic heritability.

ABSTRACT

Consider the following three important problems in statistical inference, namely, constructing confidence intervals for (1) the error of a high-dimensional ($p>n$) regression estimator, (2) the linear regression noise level, and (3) the genetic signal-to-noise ratio of a continuous-valued trait (related to the heritability). All three problems turn out to be closely related to the little-studied problem of performing inference on the $\ell_2$-norm of the signal in high-dimensional linear regression. We derive a novel procedure for this, which is asymptotically correct when the covariates are multivariate Gaussian and produces valid confidence intervals in finite samples as well. The procedure, called EigenPrism, is computationally fast and makes no assumptions on coefficient sparsity or knowledge of the noise level. We investigate the width of the EigenPrism confidence intervals, including a comparison with a Bayesian setting in which our interval is just 5% wider than the Bayes credible interval. We are then able to unify the three aforementioned problems by showing that the EigenPrism procedure with only minor modifications is able to make important contributions to all three. We also investigate the robustness of coverage and find that the method applies in practice and in finite samples much more widely than just the case of multivariate Gaussian covariates. Finally, we apply EigenPrism to a genetic dataset to estimate the genetic signal-to-noise ratio for a number of continuous phenotypes.

Motivation & Objective

• To develop a method for constructing confidence intervals for the $\ell_2$-norm of the regression coefficient vector $\bm{\beta}$ in high-dimensional linear models ($p > n$), a fundamental but underexplored inference problem.
• To unify inference on three key statistical problems: (1) regression estimation error, (2) noise level, and (3) genetic signal-to-noise ratio (heritability), all of which reduce to estimating $\theta^2 = \|\bm{\Sigma}^{1/2}\bm{\beta}\|_2^2$.
• To ensure finite-sample validity and asymptotic correctness under minimal assumptions—specifically, multivariate Gaussian covariates—while avoiding reliance on coefficient sparsity or knowledge of $\sigma^2$.
• To evaluate the width and robustness of the resulting confidence intervals, comparing them to Bayesian credible intervals and assessing performance under non-Gaussian designs.

Proposed method

• Proposes a new procedure, EigenPrism, based on the eigen-decomposition of the design matrix $\bm{X}^T\bm{X}$, using the squared projections of the response $\bm{y}$ onto the eigenvectors of $\bm{X}^T\bm{X}$.
• Constructs test statistics based on the $z_i^2 = \bm{u}_i^T\bm{y}^2$ (squared projections of $\bm{y}$ onto the $i$-th eigenvector $\bm{u}_i$) to estimate $\theta^2$.
• Uses a studentized pivot based on the asymptotic distribution of the eigenvalues and the $z_i^2$ statistics to construct two-sided confidence intervals for $\theta^2$.
• Employs a variance stabilization and bias correction technique to improve finite-sample coverage, particularly in high-dimensional settings.
• Adapts the core EigenPrism framework to three distinct inference problems by reparameterizing the signal-to-noise ratio in terms of regression error, noise variance, and heritability.
• Validates the method through extensive simulations and real data analysis on a genetic dataset (NFBC1966), demonstrating robustness beyond Gaussian design assumptions.

Experimental results

Research questions

• RQ1 Can we construct valid confidence intervals for the $\ell_2$-norm of the coefficient vector $\bm{\beta}$ in high-dimensional linear models ($p > n$) without assuming sparsity or knowing $\sigma^2$?
• RQ2 How does the coverage of the EigenPrism confidence interval perform in finite samples, especially when the design matrix is not multivariate Gaussian?
• RQ3 Can the same core procedure be adapted to unify inference on regression error, noise level, and genetic signal-to-noise ratio (heritability) in high-dimensional settings?
• RQ4 How does the width of the EigenPrism confidence interval compare to that of a Bayesian credible interval under the same model assumptions?
• RQ5 What diagnostics or robustness checks can be performed to assess the validity of the EigenPrism procedure in practice, especially when the Gaussian assumption on $\bm{X}$ is violated?

Key findings

• EigenPrism produces confidence intervals for $\theta^2$ with finite-sample coverage that is close to the nominal level, even in high-dimensional settings ($p > n$), under multivariate Gaussian design.
• The method achieves 95% confidence interval coverage in simulations and real data, with interval widths only 5% wider than the corresponding Bayesian credible intervals under the same model.
• EigenPrism is robust to deviations from multivariate Gaussian design: simulations show it maintains good coverage under various non-Gaussian and correlated designs.
• The procedure successfully unifies inference across three distinct problems: regression error, noise level, and genetic signal-to-noise ratio, with only minor modifications to the core framework.
• In the analysis of the NFBC1966 genetic dataset, EigenPrism produced 95% confidence intervals for heritability that were consistent with prior literature estimates (e.g., 73.8% and 62.5% for height), despite using a fixed-effects model rather than a random-effects model.
• The method does not require knowledge of $\sigma^2$ or sparsity assumptions, and its validity is grounded in observable design matrix properties, enabling diagnostic checks for reliability.