[Paper Review] Interpreting Event-Studies from Recent Difference-in-Differences Methods

Jonathan Roth|arXiv (Cornell University)|Jan 22, 2024

Advanced Causal Inference Techniques35 citations

TL;DR

The paper shows that event-study plots from recent DiD methods (dCDH, CS, BJS) differ structurally from traditional TWFE plots, due to asymmetric pre/post-treatment construction, and offers practical plotting recommendations.

ABSTRACT

This note discusses the interpretation of event-study plots produced by recent difference-in-differences methods. I show that even when specialized to the case of non-staggered treatment timing, the default plots produced by software for several of the most popular recent methods do not match those of traditional two-way fixed effects (TWFE) event-studies. The plots produced by the new methods may show a kink or jump at the time of treatment even when the TWFE event-study shows a straight line. This difference stems from the fact that the new methods construct the pre-treatment coefficients asymmetrically from the post-treatment coefficients. As a result, visual heuristics for evaluating violations of parallel trends using TWFE event-study plots should not be immediately applied to those from these methods. I conclude with practical recommendations for constructing and interpreting event-study plots when using these methods.

Motivation & Objective

• Motivate how modern DiD methods produce event-study plots that differ from traditional TWFE plots.
• Explain why pre-treatment and post-treatment coefficients are constructed asymmetrically in CS/dCDH and BJS.
• Demonstrate, via simulation, how these differences affect visual interpretation of event-study plots.
• Provide practical recommendations for constructing and interpreting event-study plots with CS/dCDH and BJS.

Proposed method

• Simulates a simple non-staggered DiD setting to compare TWFE, CS, dCDH, and BJS event-studies.
• Mathematically derives the pre/post-treatment coefficient constructions for TWFE, CS/dCDH, and BJS.
• Shows that CS/dCDH use short-differences before treatment and long-differences after treatment, creating a potential kink.
• Demonstrates that BJS uses asymmetric imputation and pre-treatment averaging, leading to differences from TWFE plots.
• Offers practical remedies such as using universal base period or long-differences to align plots with conventional interpretations.

(a) Dynamic TWFE

Experimental results

Research questions

• RQ1 Do event-study plots from dCDH, CS, and BJS align with traditional TWFE plots in non-staggered settings?
• RQ2 Why do modern DiD event-studies exhibit kinks or jumps near treatment dates despite parallel trends?
• RQ3 How does the construction of pre-treatment vs post-treatment coefficients differ across methods?
• RQ4 What practical steps can researchers take to render these plots more comparable and interpretable?

Key findings

• TWFE event-studies in the non-staggered setting produce a linear pre-treatment trend that continues post-treatment.
• CS and dCDH produce a kink at the treatment date due to asymmetric pre-treatment (short-differences) and post-treatment (long-differences) construction.
• BJS can show an upward pre-trend with a sharp downward jump at treatment, reflecting its imputation-based and asymmetric design.
• The asymmetry in pre/post-treatment construction explains why visual heuristics from TWFE plots mislead when applied to CS/dCDH and BJS plots.
• Using long-differences for pre-treatment coefficients (universal base period) can make CS/dCDH plots more comparable to TWFE plots.
• Pre-treatment tests of parallel trends and post-treatment effect estimates remain valid under their respective assumptions, but visualization should be interpreted cautiously.

(b) CS