[Paper Review] Two-stage differences in differences

John Gardner|arXiv (Cornell University)|Jul 13, 2022

Advanced Causal Inference Techniques301 citations

TL;DR

Introduces a two-stage estimator for staggered adoption with heterogeneous treatment effects, showing robustness where standard DiD misidentifies the average treatment effect.

ABSTRACT

A recent literature has shown that when adoption of a treatment is staggered and average treatment effects vary across groups and over time, difference-in-differences regression does not identify an easily interpretable measure of the typical effect of the treatment. In this paper, I extend this literature in two ways. First, I provide some simple underlying intuition for why difference-in-differences regression does not identify a group$ imes$period average treatment effect. Second, I propose an alternative two-stage estimation framework, motivated by this intuition. In this framework, group and period effects are identified in a first stage from the sample of untreated observations, and average treatment effects are identified in a second stage by comparing treated and untreated outcomes, after removing these group and period effects. The two-stage approach is robust to treatment-effect heterogeneity under staggered adoption, and can be used to identify a host of different average treatment effect measures. It is also simple, intuitive, and easy to implement. I establish the theoretical properties of the two-stage approach and demonstrate its effectiveness and applicability using Monte-Carlo evidence and an example from the literature.

Motivation & Objective

• Clarify why standard difference-in-differences may not identify a simple group×period average treatment effect under staggered adoption and heterogeneity.
• Propose a two-stage estimation framework that isolates group and period effects from untreated observations and then estimates the average treatment effect on the treated.
• Establish theoretical properties, including consistency and unbiasedness, and provide guidance for inference and implementation.
• Demonstrate performance via Monte Carlo simulations and an empirical application.
• Discuss extensions to event-study analyses and alternative estimands.

Proposed method

• First stage: regress outcomes on group and period fixed effects using only untreated observations to estimate λ_g and γ_p.
• Second stage: regress Y_gpit − ŀ_g − ŷ_p on D_gp to identify E(β_gp | D_gp = 1) under parallel trends.
• Interpretation: the two-stage estimator recovers the overall average treatment effect on the treated even with heterogeneity across groups and periods.
• The method can be extended to different estimands (e.g., four-period average, duration-specific effects) and to event-study settings.
• Inference can be conducted via GMM, with standard errors adjusted for generated regressors.

Experimental results

Research questions

• RQ1 What does the standard DiD estimand identify when treatment effects are heterogeneous and adoption is staggered?
• RQ2 Can a simple two-stage approach yield a robust, interpretable estimate of the average treatment effect on the treated under staggered adoption?
• RQ3 How does the two-stage estimator perform relative to other approaches (e.g., separate group×period estimates, stacked DiD) in simulations and empirical data?
• RQ4 How can the framework be extended to event studies and alternative summary measures of treatment effects?

Key findings

• Standard DiD regression does not generally identify a simple group×period average treatment effect under staggered adoption with heterogeneous effects.
• The proposed two-stage estimator identifies E(β_gp|D_gp = 1), the average treatment effect across groups and periods among the treated.
• In simulations, the two-stage estimator matches aggregated true averages and outperforms naive DiD in the presence of heterogeneity.
• In event-study contexts, the two-stage approach can recover pre-trends and the evolution of effects over the duration of treatment.
• An empirical application reproduces baseline results while illustrating how DiD weights can distort interpretation under heterogeneity.