[Paper Review] Estimating Total Treatment Effect in Randomized Experiments with Unknown Network Structure

Christina Lee Yu, Edoardo M. Airoldi+2 more|arXiv (Cornell University)|May 25, 2022

Advanced Causal Inference Techniques4 citations

TL;DR

This paper proposes a simple, unbiased estimator for total treatment effect in randomized experiments with unknown network interference, leveraging historical baseline data to eliminate bias without requiring knowledge of the underlying network structure. The key contribution is a statistically efficient, network-agnostic estimator that achieves low variance under mild regularity conditions, enabling reliable causal inference in settings with heterogeneous peer effects.

ABSTRACT

Randomized experiments are widely used to estimate the causal effects of a proposed treatment in many areas of science, from medicine and healthcare to the physical and biological sciences, from the social sciences to engineering, to public policy and to the technology industry at large. Here, we consider situations where classical methods for estimating the total treatment effect on a target population are considerably biased due to confounding network effects, i.e., the fact that the treatment of an individual may impact their neighbors' outcomes, an issue referred to as network interference or as non-individualized treatment response. A key challenge in these situations, is that the network is often unknown, and difficult, or costly, to measure. In this paper, we characterize the limitations in estimating the total treatment effect without knowledge of the network that drives interference, assuming a potential outcomes model with heterogeneous additive network effects. This model encompasses a broad class of network interference sources, including spillover, peer effects, and contagion. Within this framework, we show that, surprisingly, given access to average historical baseline measurements prior to the experiment, we can develop a simple estimator and efficient randomized design that outputs an unbiased estimate with low variance. Our solution does not require knowledge of the underlying network structure, and it comes with statistical guarantees for a broad class of models. We believe our results are poised to impact current randomized experimentation strategies due to its ease of interpretation and implementation, alongside its provable theoretical insights under heterogeneous network effects.

Motivation & Objective

• To address bias in total treatment effect estimation when network interference violates the Stable Unit Treatment Value Assumption (SUTVA) due to unobserved network structures.
• To develop a method for unbiased estimation of total treatment effect without requiring knowledge of the underlying network, even under heterogeneous additive network effects.
• To provide statistical guarantees and efficient design strategies for randomized experiments in the presence of network interference with unknown topology.
• To demonstrate that prior baseline data enables unbiased estimation without network knowledge, overcoming a fundamental limitation in existing approaches.

Proposed method

• Proposes a linear estimator, $\widehat{\text{TTE}}_{-\alpha} = \frac{1}{p}\left(\frac{1}{n}\sum_{i\in[n]}Y_{i}(\mathbf{z}) - \frac{1}{n}\sum_{i\in[n]}\alpha_{i}\right)$, where $\alpha_i$ are historical baseline estimates, to correct for network interference.
• Characterizes individual influence $L_i = \beta_i + \sum_{k\in[n]} \frac{\mathbb{E}[z_i]\gamma_{ik}}{\mathbb{E}[z_k]}$ as a function of treatment effects and network structure, enabling variance analysis.
• Uses completely randomized design (CRD) to achieve optimal efficiency, with variance bounded by $O(1/pn \cdot B^2 d_{\text{max}}^2)$ under bounded effect parameters and degrees.
• Introduces a uniform saturation design that groups individuals by covariates and local network structure to minimize estimator variance.
• Establishes that unbiased estimation is impossible without network knowledge unless the network is fully decomposable into isolated components under joint treatment.
• Derives asymptotic normality of the estimator under CRD, enabling p-values and hypothesis testing via variance estimation.

Experimental results

Research questions

• RQ1 Can we estimate the total treatment effect unbiasedly in randomized experiments when the network structure causing interference is unknown?
• RQ2 What role does historical baseline data play in enabling unbiased estimation without network knowledge?
• RQ3 How does the variance of the total treatment effect estimator depend on individual influence and network structure?
• RQ4 What randomized design minimizes variance under unknown network interference and bounded causal effects?
• RQ5 Under what conditions is it possible to achieve consistent estimation of the total treatment effect without observing the network?

Key findings

• Without prior baseline data, no unbiased linear estimator for total treatment effect exists unless the network is fully decomposable into disconnected components treated jointly.
• With access to historical baseline estimates $\alpha_i$, the proposed estimator $\widehat{\text{TTE}}_{-\alpha}$ is unbiased under any randomized design with marginal treatment probability $p$.
• The variance of the estimator is bounded by $O(1/pn \cdot B^2 d_{\text{max}}^2)$ under CRD, where $B$ bounds the causal effect parameters and $d_{\text{max}}$ bounds individual out-degrees.
• The estimator's sampling distribution is approximately Gaussian for large $n$, enabling valid inference via standard errors and p-values.
• The influence term $L_i$ quantifies individual $i$'s contribution to the total treatment effect and determines the estimator’s variance, with optimal designs balancing $L_i$ distributions across treatment and control.
• Variance can be further reduced by using a uniform saturation design that groups individuals with similar $L_i$ values based on observed covariates and local network structure.