[Paper Review] Penalized regression adjusted causal effect estimates in high dimensional randomized experiments

Hanzhong Liu, Yuehan Yang|arXiv (Cornell University)|Sep 24, 2018

Advanced Causal Inference Techniques3 citations

TL;DR

This paper establishes asymptotic normality and efficiency gains for penalized regression-adjusted average causal effect (ACE) estimators—Ridge, Elastic Net, and Adaptive Lasso—in high-dimensional randomized experiments under the Neyman-Rubin potential outcomes framework. It proves that these estimators achieve asymptotic variance no greater than the difference-in-means estimator when risk consistent, and provides conservative variance estimators for valid confidence intervals.

ABSTRACT

Regression adjustments are often considered by investigators to improve the estimation efficiency of causal effect in randomized experiments when there exists many pre-experiment covariates. In this paper, we provide conditions that guarantee the penalized regression including the Ridge, Elastic Net and Adapive Lasso adjusted causal effect estimators are asymptotic normal and we show that their asymptotic variances are no greater than that of the simple difference-in-means estimator, as long as the penalized estimators are risk consistent. We also provide conservative estimators for the asymptotic variance which can be used to construct asymptotically conservative confidence intervals for the average causal effect (ACE). Our results are obtained under the Neyman-Rubin potential outcomes model of randomized experiment when the number of covariates is large. Simulation study shows the advantages of the penalized regression adjusted ACE estimators over the difference-in-means estimator.

Motivation & Objective

• To address the inefficiency of difference-in-means estimators in high-dimensional randomized experiments with many pre-treatment covariates.
• To establish theoretical guarantees—specifically asymptotic normality and variance reduction—for penalized regression-adjusted ACE estimators under the Neyman-Rubin model.
• To provide conservative variance estimators for constructing asymptotically valid confidence intervals in high-dimensional settings.
• To extend low-dimensional regression adjustment theory to high-dimensional settings where ordinary least squares fails due to overfitting.
• To investigate the finite-sample performance and robustness of Ridge, Elastic Net, and Adaptive Lasso in causal inference under high-dimensional covariates.

Proposed method

• Uses the Neyman-Rubin potential outcomes model to define the average causal effect (ACE) as the mean difference in potential outcomes under treatment and control.
• Applies penalized regression (Ridge, Elastic Net, Adaptive Lasso) to adjust for high-dimensional covariates, replacing ordinary least squares to avoid overfitting.
• Derives conditions under which the penalized regression-adjusted ACE estimators are asymptotically normal and have variance no larger than the unadjusted difference-in-means estimator.
• Proposes conservative variance estimators based on the asymptotic variance of the penalized estimators, enabling valid confidence intervals.
• Employs bootstrap resampling (B=500) to estimate standard errors and assess coverage and interval length in simulations.
• Analyzes the impact of regularization on estimation efficiency and inference validity under finite-population sampling without replacement.

Experimental results

Research questions

• RQ1 Under what conditions are penalized regression-adjusted ACE estimators asymptotically normal in high-dimensional randomized experiments?
• RQ2 Can penalized regression estimators achieve asymptotic variance no greater than the difference-in-means estimator under risk consistency?
• RQ3 Do conservative variance estimators derived from the asymptotic variance of penalized estimators yield asymptotically conservative confidence intervals?
• RQ4 How do Ridge, Elastic Net, and Adaptive Lasso perform relative to unadjusted and OLS estimators in terms of bias, variance, and coverage in high-dimensional settings?
• RQ5 What is the impact of model misspecification (e.g., non-linear relationships, fixed covariates) on the performance of penalized regression adjustments in causal inference?

Key findings

• Penalized regression-adjusted ACE estimators (Ridge, Elastic Net, Adaptive Lasso) are asymptotically normal under mild regularity conditions when the estimators are risk consistent.
• The asymptotic variance of all penalized estimators is no greater than that of the difference-in-means estimator, confirming improved efficiency.
• Conservative variance estimators were derived and shown to yield asymptotically conservative confidence intervals, improving coverage in finite samples.
• Simulation results show that Lasso, Elastic Net, and Adaptive Lasso estimators achieve significantly lower mean squared error (MSE) and variance than the unadjusted difference-in-means estimator, especially in high-dimensional settings.
• The naive Elastic Net estimator (without tuning) showed poor coverage (e.g., 52.3% at (500,120)), while properly tuned versions (e.g., EN, Ada) maintained coverage above 90%.
• Ridge and Adaptive Lasso estimators consistently outperformed unadjusted and OLS estimators in terms of MSE and interval length, with MSE reduced by up to 70% in some scenarios.