[Paper Review] Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey

Clément de Chaisemartin, Xavier D’Haultfœuille|arXiv (Cornell University)|Dec 8, 2021

Advanced Causal Inference Techniques107 citations

TL;DR

A survey of TWFE and DiD with heterogeneous treatment effects, detailing identification issues, diagnostic tools, and robust alternative estimators, plus a re-analysis of Wolfers (2006a).

ABSTRACT

Linear regressions with period and group fixed effects are widely used to estimate policies' effects: 26 of the 100 most cited papers published by the American Economic Review from 2015 to 2019 estimate such regressions. It has recently been shown that those regressions may produce misleading estimates, if the policy's effect is heterogeneous between groups or over time, as is often the case. This survey reviews a fast-growing literature that documents this issue, and that proposes alternative estimators robust to heterogeneous effects. We use those alternative estimators to revisit Wolfers (2006).

Motivation & Objective

• Assess robustness of TWFE and DiD estimators under treatment effect heterogeneity.
• Summarize diagnostic tools and decomposition results that reveal potential biases.
• Present alternative estimators that are robust to heterogeneous effects.
• Revisit Wolfers (2006a) in light of new methodological insights.
• Provide practical guidance and software references for practitioners.

Proposed method

• Review and synthesize the literature on TWFE regressions with heterogeneous treatment effects.
• Derive and present decomposition results showing how TWFE weights can be negative and need not imply a convex combination of effects.
• Explain the origin of the problem via forbidden comparisons in binary and staggered designs, and extend to non-binary/non-staggered cases.
• Discuss event-study (Sun and Abraham) decompositions for dynamic TWFE regressions and potential contamination from other treatment periods.
• Present diagnostics and commands (twowayfeweights, bacondecomp) used to assess weights and DID components.

Experimental results

Research questions

• RQ1 Do TWFE regressions identify a convex combination of heterogeneous treatment effects under common parallel trends?
• RQ2 Under heterogeneous effects, how do TWFE weights distribute attention across treatment cells and can they be negative?
• RQ3 When do “forbidden comparisons” arise, and how do they bias TWFE estimates in binary/staggered vs non-binary/non-staggered designs?
• RQ4 What diagnostic tools and alternative estimators robust to heterogeneity exist, and how do they perform in practice (e.g., re-analysis of Wolfers 2006a)?

Key findings

• TWFE estimators may not identify a convex combination of treatment effects when effects are heterogeneous across groups or over time.
• Weights in the TWFE decomposition can be negative, leading to sign reversals and biased ATT estimates in many realistic designs.
• In binary-staggered designs, some decompositions show that TWFE is a weighted average of DIDs with potentially non-convex weights due to forbidden comparisons.
• Diagnostic tools (twowayfeweights, bacondecomp) reveal the distribution and sign of weights and the presence of negative contributions.
• Event-study based TWFE decompositions (Sun and Abraham) show potential contamination from varying effects across lags, affecting interpretation.