[Paper Review] Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models

Alexandre Belloni, Victor Chernozhukov+1 more|arXiv (Cornell University)|Dec 27, 2013

Statistical Methods and Inference4 citations

TL;DR

This paper proposes valid post-selection inference methods for high-dimensional quantile regression models under approximate sparsity, using orthogonal score functions to ensure robustness against model selection errors. It establishes uniform asymptotic normality and valid confidence regions for regression coefficients, even when the number of regressors exceeds sample size, without requiring beta-min conditions.

ABSTRACT

This work proposes new inference methods for a regression coefficient of interest in a (heterogeneous) quantile regression model. We consider a high-dimensional model where the number of regressors potentially exceeds the sample size but a subset of them suffice to construct a reasonable approximation to the conditional quantile function. The proposed methods are (explicitly or implicitly) based on orthogonal score functions that protect against moderate model selection mistakes, which are often inevitable in the approximately sparse model considered in the present paper. We establish the uniform validity of the proposed confidence regions for the quantile regression coefficient. Importantly, these methods directly apply to more than one variable and a continuum of quantile indices. In addition, the performance of the proposed methods is illustrated through Monte-Carlo experiments and an empirical example, dealing with risk factors in childhood malnutrition.

Motivation & Objective

• To develop inference methods for regression coefficients in high-dimensional quantile regression models where p ≫ n.
• To ensure validity of confidence regions despite model selection errors common in high-dimensional settings.
• To extend inference to multivariate treatments and a continuum of quantile indices.
• To remove the need for beta-min conditions by leveraging orthogonal score functions.
• To provide uniformly valid confidence regions under weak regularity and sparsity assumptions.

Proposed method

• Uses orthogonal score functions that are robust to first-order estimation errors in the confounding function gτ.
• Employs ℓ1-penalized quantile regression and post-selection estimation for initial model fitting.
• Applies heteroscedastic post-Lasso to a density-weighted equation for partialling out confounders.
• Constructs two estimators: one via Neyman-type score statistic and another via density-weighted quantile regression on selected variables.
• Implements a pivotal linear representation for the estimator to enable valid inference.
• Uses the asymptotic chi-squared distribution of the Neyman-type score statistic for confidence band construction.

Experimental results

Research questions

• RQ1 Can valid confidence regions be constructed for quantile regression coefficients after model selection in high-dimensional approximately sparse models?
• RQ2 How can inference remain valid when model selection errors are inevitable in high-dimensional settings?
• RQ3 Can the proposed method handle multivariate treatments and a continuum of quantile indices?
• RQ4 Is the inference procedure robust to non-regular estimation of the confounding function gτ?
• RQ5 Does the method avoid requiring beta-min conditions for asymptotic validity?

Key findings

• The proposed estimator is root-n consistent and asymptotically normal, with a pivotal linear representation.
• Confidence regions based on the estimated standard error achieve asymptotic coverage probability 1−ξ under mild moment and sparsity conditions.
• The Neyman-type score statistic is asymptotically chi-squared with one degree of freedom under the null, enabling valid confidence bands.
• The method achieves uniform validity over a broad class of data-generating processes under array asymptotics.
• The confidence regions remain valid without requiring separation of regression coefficients from zero (no beta-min condition).
• Monte Carlo experiments and empirical application to childhood malnutrition risk factors confirm the method's finite-sample performance.