[Paper Review] Subgroup Difference in Differences to Identify Effect Modification Without a Control Group

Zach Shahn|arXiv (Cornell University)|Jun 19, 2023

Advanced Causal Inference TechniquesMathematics3 citations

TL;DR

This paper proposes a subgroup difference-in-differences (SDiD) method to identify effect modification by baseline covariates—such as sex or race—without requiring a control group. Under the subgroup parallel trends assumption, the difference in pre-post outcome changes between subgroups identifies the difference in treatment effects across those subgroups, even when individual subgroup effects are not separately identifiable.

ABSTRACT

Suppose it is of interest to characterize effect heterogeneity of an intervention across levels of a baseline covariate using only pre- and post- intervention outcome measurements from those who received the intervention, i.e. with no control group. For example, a researcher concerned with equity may wish to ascertain whether a minority group benefited less from an intervention than the majority group. We introduce the `subgroup parallel trends' assumption that the counterfactual untreated outcomes in each subgroup of interest follow parallel trends pre- and post- intervention. Under the subgroup parallel trends assumption, it is straightforward to show that a simple `subgroup difference in differences' (SDiD) expression (i.e., the average pre/post outcome difference in one subgroup subtracted by the average pre/post outcome difference in the other subgroup) identifies the difference between the intervention's effects in the two subgroups. This difference in effects across subgroups is identified even though the conditional effects in each subgroup are not. The subgroup parallel trends assumption is not stronger than the standard parallel trends assumption across treatment groups when a control group is available, and there are circumstances where it is more plausible. Thus, when effect modification by a baseline covariate is of interest, researchers might consider SDiD whether or not a control group is available.

Motivation & Objective

• To develop a method for identifying effect modification by baseline covariates when no control group is available.
• To enable causal inference about heterogeneous treatment effects in settings with only treated units and pre-post data.
• To support equity-focused analyses where it is critical to assess whether interventions benefit subgroups unequally.
• To provide a valid alternative to traditional DiD when standard parallel trends across groups fails but subgroup trends remain parallel.
• To formalize and validate a commonly used empirical approach—comparing pre-post changes across subgroups—under a clear identifying assumption.

Proposed method

• Proposes a subgroup difference-in-differences (SDiD) estimator that compares pre-post outcome changes across subgroups defined by baseline covariates.
• Relies on the subgroup parallel trends assumption: the counterfactual untreated outcome trends are parallel across subgroups defined by X.
• Uses the expression $ E[Y_1 - Y_0 | X = x] - E[Y_1 - Y_0 | X = x'] $ to identify the difference in treatment effects between subgroups.
• Derives identification under consistency and the subgroup parallel trends assumption, showing that the SDiD estimator equals the causal estimand of interest.
• Applies plug-in estimation via sample averages for categorical X or regression models for continuous X.
• Highlights that the method remains valid even if pre-post comparisons within subgroups are biased, as long as subgroup trends are parallel.

Experimental results

Research questions

• RQ1 Can we identify effect modification by baseline covariates in the absence of a control group using only pre-post data from treated units?
• RQ2 Under what assumption does the difference in pre-post outcome changes between subgroups identify the difference in treatment effects across those subgroups?
• RQ3 Is the SDiD estimator valid even when individual subgroup treatment effects are not separately identifiable?
• RQ4 In what scenarios might the subgroup parallel trends assumption hold when the standard parallel trends assumption fails?
• RQ5 Can SDiD be used to test theoretical predictions about heterogeneous policy impacts across subgroups?

Key findings

• The SDiD estimator identifies the difference in treatment effects between subgroups under the subgroup parallel trends assumption, even without a control group.
• The method allows for valid inference on effect modification in equity-focused or theory-testing contexts where subgroup fairness or differential impact is central.
• The subgroup parallel trends assumption is stronger than no assumption but no stronger than the standard DiD parallel trends assumption across groups.
• SDiD may be preferred over traditional DiD when external time-varying confounders affect the treated group uniformly across subgroups.
• The approach remains valid even if pre-post comparisons within subgroups are biased, as long as counterfactual trends are parallel across subgroups.
• The method provides a practical alternative when control groups are unavailable and standard DiD assumptions are violated.