[Paper Review] Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects

Liyang Sun, Sarah Abraham|arXiv (Cornell University)|Apr 16, 2018

Advanced Causal Inference Techniques17 references186 citations

TL;DR

The paper shows that two-way fixed effects with leads and lags can be contaminated by treatment-effect heterogeneity across adoption cohorts, and it proposes a robust alternative estimator and a weights-based diagnostic tool.

ABSTRACT

To estimate the dynamic effects of an absorbing treatment, researchers often use two-way fixed effects regressions that include leads and lags of the treatment. We show that in settings with variation in treatment timing across units, the coefficient on a given lead or lag can be contaminated by effects from other periods, and apparent pretrends can arise solely from treatment effects heterogeneity. We propose an alternative estimator that is free of contamination, and illustrate the relative shortcomings of two-way fixed effects regressions with leads and lags through an empirical application.

Motivation & Objective

• Identify pitfalls in interpreting dynamic treatment effects from two-way fixed effects with staggered adoption across cohorts.
• Decompose the leading FE estimator to show contamination from heterogeneity across relative periods.
• Propose an alternative regression-based estimator that remains interpretable under heterogeneous treatment effects.
• Provide a method to compute cohort-specific weights and assess their impact on estimated dynamics.
• Illustrate theoretical results with an empirical hospitalization study.

Proposed method

• Formalize the event-study design with absorbing treatment and cohort classification by initial treatment time.
• Derive the decomposition of the FE relative-period coefficients into cohort-specific treatment effects across all relative periods.
• Show that coefficients are linear combinations of cohort-specific average treatment effects on the treated (CATT) and other cohorts’ effects.
• Introduce an auxiliary regression to estimate the weights governing the linear combination of underlying treatment effects.
• Propose an alternative estimator that yields a weighted average of treatment effects with interpretable cohort shares, allowing covariates as in Callaway and Sant’Anna (2020a).
• Provide an empirical illustration using hospitalization effects on earnings and out-of-pocket spending.

Experimental results

Research questions

• RQ1 How do heterogeneous treatment effects across cohorts affect the interpretation of dynamic treatment effect estimates from two-way FE regressions?
• RQ2 Can we quantify and diagnose the contamination of FE-based estimates due to heterogeneity, and how can we construct alternative estimators that yield interpretable causal dynamics?
• RQ3 What is the role of cohort-specific weights in shaping the FE estimates, and how can we compute them?
• RQ4 How does an alternative regression-based approach perform in practice compared to standard FE in dynamic event-study settings?
• RQ5 Can the proposed method accommodate covariates and still provide interpretable, weighted averages of treatment effects?

Key findings

• FE estimates of dynamic effects can be contaminated by treatment effects from other relative periods when treatment timing varies across units.
• The coefficients on leads/lags in FE regressions are linear combinations of cohort-specific treatment effects, not pure period-specific effects, unless strong homogeneity holds.
• Standard pretrends tests based on lead coefficients can be invalid under treatment-effect heterogeneity.
• An auxiliary regression can compute the weights that determine how much each cohort and relative period contributes to a FE coefficient.
• The paper proposes a regression-based alternative estimator that produces a weighted average of treatment effects with interpretable cohort shares, and it can incorporate covariates.
• An empirical application to hospitalization data shows the alternative method yields similar broad conclusions to FE for earnings decline, but with estimates constrained to lie within the convex hull of underlying effects, unlike some FE estimates.