[Paper Review] Regression adjustments for estimating the global treatment effect in experiments with interference

Alex Chin|arXiv (Cornell University)|Aug 27, 2018

Advanced Causal Inference Techniques79 references3 citations

TL;DR

This paper proposes regression adjustment estimators for unbiased estimation of the global average treatment effect (GATE) in randomized experiments with interference, using treatment assignment functions as covariates and enabling flexible modeling via machine learning. The method improves bias correction over existing inverse propensity weighting and enables valid inference through bootstrap resampling under exogeneity assumptions.

ABSTRACT

Standard estimators of the global average treatment effect can be biased in the presence of interference. This paper proposes regression adjustment estimators for removing bias due to interference in Bernoulli randomized experiments. We use a fitted model to predict the counterfactual outcomes of global control and global treatment. Our work differs from standard regression adjustments in that the adjustment variables are constructed from functions of the treatment assignment vector, and that we allow the researcher to use a collection of any functions correlated with the response, turning the problem of detecting interference into a feature engineering problem. We characterize the distribution of the proposed estimator in a linear model setting and connect the results to the standard theory of regression adjustments under SUTVA. We then propose an estimator that allows for flexible machine learning estimators to be used for fitting a nonlinear interference functional form. We propose conducting statistical inference via bootstrap and resampling methods, which allow us to sidestep the complicated dependences implied by interference and instead rely on empirical covariance structures. Such variance estimation relies on an exogeneity assumption akin to the standard unconfoundedness assumption invoked in observational studies. In simulation experiments, our methods are better at debiasing estimates than existing inverse propensity weighted estimators based on neighborhood exposure modeling. We use our method to reanalyze an experiment concerning weather insurance adoption conducted on a collection of villages in rural China.

Motivation & Objective

• To address bias in global average treatment effect (GATE) estimation caused by interference in randomized experiments.
• To extend regression adjustment techniques—commonly used under SUTVA—to settings with interference by constructing covariates from treatment assignment functions.
• To enable flexible, nonlinear modeling of interference effects using machine learning estimators for the mean function.
• To develop valid statistical inference under exogeneity assumptions, avoiding reliance on complex analytic variance formulas.
• To demonstrate improved bias reduction compared to existing inverse propensity weighting methods in simulations and real-world data.

Proposed method

• Constructs regression adjustment covariates as functions of the treatment assignment vector, capturing interference patterns through user-defined functions correlated with the response.
• Uses a linear model framework to characterize the distribution of the proposed estimator and connects it to standard regression adjustment theory under SUTVA.
• Proposes a flexible estimator using machine learning models to fit nonlinear interference functional forms, improving model fit in complex settings.
• Employs bootstrap and resampling methods for variance estimation, leveraging empirical covariance structures to handle dependence induced by interference.
• Relies on an exogeneity assumption akin to unconfoundedness in observational studies, ensuring valid inference when unmeasured confounders are absent.
• Applies the method to real data from a rural China weather insurance experiment, demonstrating practical utility.

Experimental results

Research questions

• RQ1 Can regression adjustments effectively reduce bias in GATE estimation when interference violates SUTVA?
• RQ2 How can treatment assignment functions be used as covariates to model interference in a way that generalizes standard regression adjustments?
• RQ3 What is the performance of bootstrap-based inference for GATE estimators under interference, compared to analytical variance formulas?
• RQ4 How does the proposed method compare to inverse propensity weighting with exposure models in terms of bias and precision?
• RQ5 Can machine learning models improve estimation accuracy in settings with complex, nonlinear interference patterns?

Key findings

• The proposed regression adjustment estimators significantly reduce bias compared to inverse propensity weighted estimators in simulation experiments.
• In the reanalysis of a rural China weather insurance adoption experiment, the linear and logistic regression-adjusted GATE estimates were 0.1218 and 0.1197, respectively, with standard errors of 0.0561 and 0.0559.
• Standard error estimates were wide, suggesting caution in interpreting the treatment effect, and conservative variance estimators for Hájek estimators were found to be unstable and often greater than 1.
• The method enables valid inference via bootstrap, sidestepping the need for complex analytic variance calculations under interference.
• The approach treats interference detection as a feature engineering problem, allowing researchers to incorporate domain knowledge through function selection.
• Residual-based diagnostics using cluster bootstrap or robust standard errors may help detect remaining interference not captured in the mean model.