[Paper Review] Image-based Treatment Effect Heterogeneity

Connor Jerzak, Fredrik Johansson+1 more|arXiv (Cornell University)|Jun 13, 2022

Advanced Causal Inference Techniques4 citations

TL;DR

This paper proposes a probabilistic image-type clustering model to identify treatment effect heterogeneity in images using variational Bayesian inference. By modeling image-derived clusters that predict heterogeneous causal effects, it enables interpretable, uncertainty-aware salience analysis and out-of-sample policy targeting, demonstrating improved detection of heterogeneity in a Ugandan anti-poverty RCT compared to tabular-only methods.

ABSTRACT

Randomized controlled trials (RCTs) are considered the gold standard for estimating the average treatment effect (ATE) of interventions. One use of RCTs is to study the causes of global poverty -- a subject explicitly cited in the 2019 Nobel Memorial Prize awarded to Duflo, Banerjee, and Kremer "for their experimental approach to alleviating global poverty." Because the ATE is a population summary, anti-poverty experiments often seek to unpack the effect variation around the ATE by conditioning (CATE) on tabular variables such as age and ethnicity that were measured during the RCT data collection. Although such variables are key to unpacking CATE, using only such variables may fail to capture historical, geographical, or neighborhood-specific contributors to effect variation, as tabular RCT data are often only observed near the time of the experiment. In global poverty research, when the location of the experiment units is approximately known, satellite imagery can provide a window into such factors important for understanding heterogeneity. However, there is no method that specifically enables applied researchers to analyze CATE from images. In this paper, using a deep probabilistic modeling framework, we develop such a method that estimates latent clusters of images by identifying images with similar treatment effects distributions. Our interpretable image CATE model also includes a sensitivity factor that quantifies the importance of image segments contributing to the effect cluster prediction. We compare the proposed methods against alternatives in simulation; also, we show how the model works in an actual RCT, estimating the effects of an anti-poverty intervention in northern Uganda and obtaining a posterior predictive distribution over effects for the rest of the country where no experimental data was collected. We make all models available in open-source software.

Motivation & Objective

• To address the lack of methods for analyzing causal effect heterogeneity using unstructured image data in RCTs.
• To model image-derived clusters that predict heterogeneous treatment effects, improving interpretability and scientific insight.
• To enable uncertainty quantification in cluster assignments and effect predictions using Bayesian inference.
• To develop salience measures that identify image regions most influential in determining predicted treatment effect clusters.
• To extend causal inference beyond tabular covariates by incorporating satellite imagery as a source of effect-modifying information.

Proposed method

• Proposes a probabilistic Image-Type Effect Cluster Model where image type $Z_i$ generates a distribution over treatment effects $\tau(z)$, with $\tau(z) = \mu_{\tau,z}$ and variance $\sigma_0^2 + \sigma_1^2 + \sigma_{\tau}^2$.
• Uses variational Bayesian inference to approximate the posterior $p(\mathbf{Z}, \boldsymbol{\Theta} \mid \mathbf{D})$, maximizing the Evidence Lower Bound (ELBO) with stochastic gradient descent.
• Applies reparameterization gradients to backpropagate through discrete latent variables (image clusters), enabling differentiable inference.
• Employs Monte Carlo approximation to compute expected salience: $s_{whk}^{\text{Direction}} = \sum_c \mathbb{E}\left[ \frac{\partial \text{Pr}(Z_i = k \mid M_i = m)}{\partial m_{whc}} \right]$.
• Introduces a magnitude-based salience measure $s_{whk}^{\text{Magnitude}} = \sum_c \left( \frac{\partial \mathbb{E}[\text{Pr}(Z_i = k \mid M_i = m)]}{\partial m_{whc}} \right)^2$ to identify high-impact image regions.
• Generates out-of-sample predictive distributions over treatment effects using $p(\tau_i^{\text{Out}} \mid M_i^{\text{Out}}, \mathbf{D}) = \sum_z \int p(\tau_i \mid Z_i^\text{Out}=z; \boldsymbol{\theta}) p(Z_i^\text{Out}=z \mid M_i^\text{Out}; \boldsymbol{\theta}) p(\boldsymbol{\theta} \mid \mathbf{D}) d\boldsymbol{\theta}$.

Experimental results

Research questions

• RQ1 Can image-based data reveal previously undetected treatment effect heterogeneity in RCTs beyond tabular covariates?
• RQ2 How can we model image-derived clusters to summarize and interpret heterogeneous treatment effects in a probabilistic and interpretable way?
• RQ3 What image regions most influence the predicted assignment to a treatment effect cluster, and how can we quantify their influence?
• RQ4 How does uncertainty in cluster assignment affect the reliability of salience and policy recommendations?
• RQ5 Can the model generalize to out-of-sample images and support policy targeting in new geographic or contextual settings?

Key findings

• The model successfully identifies image-based clusters that predict heterogeneous treatment effects in a Ugandan anti-poverty RCT, revealing heterogeneity not captured by tabular covariates.
• Salience analysis identifies specific image regions—particularly in land use and infrastructure features—that significantly influence predicted treatment effect clusters.
• The correlation between cluster probabilities and tabular CATEs remains high (0.85) after orthogonalization, indicating that satellite images provide independent, non-redundant information about heterogeneity.
• Posterior predictive mean cluster probabilities for the entire country reveal spatial patterns of treatment effect heterogeneity, enabling policy-relevant targeting at scale.
• The model’s uncertainty-aware salience measures are not feasible with post-hoc clustering methods due to the lack of differentiable cluster assignments.
• The approach enables out-of-sample policy targeting by forming predictive distributions over treatment effects for new images, even without tabular covariates.