[Paper Review] Difference-in-Differences Estimators When No Unit Remains Untreated

Clément de Chaisemartin, Ciccia, Diego+2 more|arXiv (Cornell University)|May 7, 2024

Optimal Experimental Design MethodsDecision Sciences3 citations

TL;DR

This paper develops robust estimators for difference-in-differences (DID) settings where no unit remains untreated, addressing bias in two-way fixed effects (TWFE) estimators under treatment effect heterogeneity. It proposes a testable condition for TWFE consistency, introduces Wald-DID and kernel-based estimators when stayers or quasi-stayers exist, and provides nonparametric bounds and a parametric estimator without such units, demonstrating that TWFE estimates in Enikolopov et al. (2011) are unreliable while those in Pierce and Schott (2016) are more robust.

ABSTRACT

We consider treatment-effect estimation with a two-periods panel, where units are untreated at period one, and receive strictly positive doses at period two. First, we consider designs with some quasi-untreated units, with a period-two dose local to zero. We show that under a parallel-trends assumption, a weighted average of slopes of units' potential outcomes is identified by a difference-in-difference estimand using quasi-untreated units as the control group. We leverage results from the regression-discontinuity-design literature to propose a nonparametric estimator. Then, we propose estimators for designs without quasi-untreated units. Finally, we propose a test of the homogeneous-effect assumption underlying two-way-fixed-effects regressions.

Motivation & Objective

• To address the inconsistency of two-way fixed effects (TWFE) estimators in heterogeneous adoption designs (HADs) where no unit remains untreated.
• To identify conditions under which TWFE remains valid, particularly mean independence of treatment effects from treatment dose.
• To develop robust estimators when TWFE assumptions fail, including Wald-DID, kernel-based, and nonparametric bound estimators.
• To re-evaluate prior empirical findings from Pierce and Schott (2016) and Enikolopov et al. (2011) using the proposed methods.
• To provide a heteroscedasticity-robust, tuning-parameter-free test for linearity of conditional expectations in univariate regressions.

Proposed method

• Proposes a test for the linearity of the conditional expectation $ E(\Delta Y_g \mid D_{g,2}) = \alpha_0 + \alpha_1 D_{g,2} $, which ensures TWFE consistency under parallel trends.
• Uses a nonparametric, heteroscedasticity-robust test based on Yatchew (1997), with asymptotic validity and no tuning parameters.
• When the test rejects linearity, proposes a Wald-DID estimator using stayers (D_{g,2}=0) as a control group, estimable via 2SLS.
• For quasi-stayers (D_{g,2} near zero), proposes a kernel-based DID estimator with an optimal bandwidth minimizing asymptotic MSE.
• In absence of stayers or quasi-stayers, derives nonparametric bounds on average treatment effects using moment inequalities.
• Proposes a parametric estimator assuming $ E[\Delta_2 \mid D_2 = d] = \delta_0 + \delta_1 d $, enabling estimation under restricted heterogeneity.

Experimental results

Research questions

• RQ1 Under what condition is the TWFE estimator consistent in HADs with no untreated units?
• RQ2 How can one test the validity of the mean independence assumption underlying TWFE consistency?
• RQ3 What robust estimators are available when the mean independence assumption fails, depending on the presence of stayers or quasi-stayers?
• RQ4 How do nonparametric bounds and parametric heterogeneity models perform when no stayers or quasi-stayers exist?
• RQ5 Are the TWFE estimates in Pierce and Schott (2016) and Enikolopov et al. (2011) robust to treatment effect heterogeneity?

Key findings

• The test for linearity of $ E(\Delta Y_g \mid D_{g,2}) $ is valid, robust to heteroscedasticity, and does not distort inference when used as a pre-test.
• For the SPS voting rate, the nonparametric lower and upper bounds on the average treatment effect are $ \widehat{B}_- = -2.88 $ and $ \widehat{B}_+ = 9.99 $, so zero cannot be ruled out.
• For the Yabloko voting rate, the bounds also include zero, and the nonparametric estimator $ \widehat{\beta}^{ns} $ is 12.78 with a large standard error (4.51), indicating high noise.
• The nonparametric estimator $ \widehat{\beta}^{ns} $ is significantly different from zero for Yabloko (p < 0.05) but not for SPS or KPRF, while TWFE estimates are insignificant for all except SPS and Yabloko.
• For LDPR vote and turnout, $ \widehat{\beta}^{ns} $ is large and negative (-39.18 and -28.19), suggesting possible model misspecification or violation of assumptions.
• The authors conclude that TWFE estimates in Enikolopov et al. (2011) are not robust to heterogeneity, while those in Pierce and Schott (2016) appear more reliable.