[Paper Review] A Unified Theory of Confidence Regions and Testing for High Dimensional Estimating Equations

Matey Neykov, Ning Yang+2 more|arXiv (Cornell University)|Oct 30, 2015

Statistical Methods and Inference4 citations

TL;DR

This paper proposes a likelihood-free inferential framework for constructing confidence regions and conducting hypothesis tests in high-dimensional models defined by estimating equations. By projecting fitted estimating equations onto a sparse direction via a large-scale linear program, it establishes a unified Z-estimation theory that enables valid inference for high-dimensional problems without requiring full likelihood specification, including in noisy compressed sensing, instrumental variables regression, graphical models, and vector autoregression.

ABSTRACT

We propose a new inferential framework for constructing confidence regions and testing hypotheses in statistical models specified by a system of high dimensional estimating equations. We construct an influence function by projecting the fitted estimating equations to a sparse direction obtained by solving a large-scale linear program. Our main theoretical contribution is to establish a unified Z-estimation theory of confidence regions for high dimensional problems. Different from existing methods, all of which require the specification of the likelihood or pseudo-likelihood, our framework is likelihood-free. As a result, our approach provides valid inference for a broad class of high dimensional constrained estimating equation problems, which are not covered by existing methods. Such examples include, noisy compressed sensing, instrumental variable regression, undirected graphical models, discriminant analysis and vector autoregressive models. We present detailed theoretical results for all these examples. Finally, we conduct thorough numerical simulations, and a real dataset analysis to back up the developed theoretical results.

Motivation & Objective

• Address the lack of valid inference methods for high-dimensional models defined by estimating equations when the dimension d exceeds sample size n.
• Overcome limitations of existing likelihood-based methods that require full distributional assumptions.
• Develop a unified framework applicable to a broad class of high-dimensional constrained estimating equation problems.
• Enable post-regularization inference for parameters of interest in models such as instrumental variables, graphical models, and vector autoregressions.
• Establish theoretical guarantees for confidence regions and hypothesis tests under minimal moment conditions, without assuming full likelihood structure.

Proposed method

• Formulate inference via a Z-estimation framework based on high-dimensional estimating equations.
• Construct an influence function by projecting the estimating equations onto a sparse direction obtained through solving a large-scale linear program.
• Use a Dantzig-selector-type constraint to enforce sparsity in the parameter estimate: minimize ℓ1 norm subject to ∥t(Z,β)∥∞ ≤ λ.
• Derive asymptotic normality of the influence function under regularity conditions, enabling confidence region construction.
• Employ concentration inequalities and random matrix theory to control estimation error and ensure consistency of the influence function.
• Establish theoretical validity of confidence regions and hypothesis tests through uniform convergence and sparsity-based bounds on estimation error.

Experimental results

Research questions

• RQ1 Can valid confidence regions be constructed for high-dimensional parameters of interest without assuming a full likelihood model?
• RQ2 How can inference be consistently performed in high-dimensional models defined by moment conditions, such as instrumental variables or graphical models?
• RQ3 What is the theoretical justification for using a sparse projection of estimating equations to construct influence functions in high-dimensional settings?
• RQ4 How does the proposed method maintain valid coverage for confidence regions when d ≫ n and the true parameter is sparse?
• RQ5 Can the framework be applied uniformly across diverse models like compressed sensing, IV regression, and vector autoregression without model-specific likelihood assumptions?

Key findings

• The proposed method achieves asymptotic normality of the influence function under mild moment conditions, enabling valid confidence regions for high-dimensional parameters.
• The framework provides valid inference for a broad class of models, including noisy compressed sensing, instrumental variables regression, undirected graphical models, discriminant analysis, and vector autoregressive models.
• Theoretical guarantees are established under sparsity and restricted eigenvalue-type conditions, ensuring convergence of the estimator and influence function.
• The method achieves uniform convergence of the empirical estimating equation to its expectation, with error bounds scaling as O(√(log d / n)) under appropriate regularity.
• Numerical simulations and real data analysis confirm the method's empirical validity and robustness across diverse high-dimensional settings.
• The approach is likelihood-free and only requires moment conditions, making it applicable to models where full likelihood is intractable or misspecified.