[Paper Review] Confidence Intervals for Low-Dimensional Parameters With High-Dimensional Data

Cun‐Hui Zhang, Stephanie S. Zhang|arXiv (Cornell University)|Oct 12, 2011

Statistical Methods and Inference27 references29 citations

TL;DR

This paper proposes a method for constructing confidence intervals for low-dimensional parameters—such as individual coefficients or linear combinations—in high-dimensional linear regression models. By establishing asymptotic normality and consistent covariance estimation under conditions where the number of predictors exceeds sample size, the approach ensures accurate coverage probabilities, validated through simulations.

ABSTRACT

Abstract. The purpose of this paper is to propose methodologies for statistical inference of low-dimensional parameters with high-dimensional data. We focus on constructing con-fidence intervals for individual coefficients and linear combinations of several of them in a linear regression model, although our ideas are applicable in a much broader context. The theoretical results presented here provide sufficient conditions for the asymptotic normality of the proposed estimators along with a consistent estimator for their finite-dimensional covariance matrices. These sufficient conditions allow the number of variables to far ex-ceed the sample size. The simulation results presented here demonstrate the accuracy of the coverage probability of the proposed confidence intervals, strongly supporting the theoretical results.

Motivation & Objective

• Address the challenge of statistical inference for low-dimensional parameters when the number of predictors far exceeds the sample size.
• Develop a method to construct valid confidence intervals for individual regression coefficients and linear combinations in high-dimensional settings.
• Establish sufficient conditions for the asymptotic normality of estimators in high-dimensional models.
• Provide a consistent estimator for the finite-dimensional covariance matrix of the estimators.
• Ensure reliable inference even when p ≫ n, extending classical inference to high-dimensional regimes.

Proposed method

• Propose a debiased or desparsified estimator for low-dimensional parameters in high-dimensional linear models.
• Derive sufficient conditions under which the proposed estimator is asymptotically normal, even when p ≫ n.
• Construct a consistent estimator for the covariance matrix of the low-dimensional parameters using high-dimensional data.
• Utilize a two-step estimation procedure: first estimate the high-dimensional regression coefficients, then correct for bias in the low-dimensional parameters.
• Apply the resulting asymptotic normality to construct confidence intervals with guaranteed coverage properties.
• Validate the method through simulation studies under varying high-dimensional settings.

Experimental results

Research questions

• RQ1 Can valid confidence intervals be constructed for individual regression coefficients when the number of predictors exceeds the sample size?
• RQ2 What conditions ensure the asymptotic normality of low-dimensional parameter estimators in high-dimensional models?
• RQ3 How can the finite-dimensional covariance matrix of low-dimensional parameters be consistently estimated in high-dimensional settings?
• RQ4 What is the empirical coverage probability of the proposed confidence intervals under high-dimensional data?
• RQ5 Can the method be extended beyond individual coefficients to linear combinations of parameters in high-dimensional models?

Key findings

• The proposed estimator for low-dimensional parameters is asymptotically normal under sufficient conditions that allow p ≫ n.
• A consistent estimator for the finite-dimensional covariance matrix of the parameters is derived, enabling valid inference.
• Simulation results show that the confidence intervals achieve coverage probabilities close to the nominal level, supporting theoretical claims.
• The method maintains accurate coverage even when the number of predictors greatly exceeds the sample size.
• The approach is applicable beyond individual coefficients to linear combinations of parameters in high-dimensional regression models.
• The theoretical framework provides a foundation for inference in high-dimensional models where classical methods fail.