[Paper Review] Difference-in-Differences when Parallel Trends Holds Conditional on Covariates

Carolina Caetano, Brantly Callaway|arXiv (Cornell University)|Jun 21, 2024

Economics of Agriculture and Food Markets11 citations

TL;DR

The paper analyzes difference-in-differences with covariates that vary over time, showing TWFE regressions can be biased under conditional parallel trends and proposing robust alternatives.

ABSTRACT

In this paper, we study difference-in-differences identification and estimation strategies when the parallel trends assumption holds after conditioning on covariates. We consider empirically relevant settings where the covariates can be time-varying, time-invariant, or both. We uncover a number of weaknesses of commonly used two-way fixed effects (TWFE) regressions in this context, even in applications with only two time periods. In addition to some weaknesses due to estimating linear regression models that are similar to cases with cross-sectional data, we also point out a collection of additional issues that we refer to as extit{hidden linearity bias} that arise because the transformations used to eliminate the unit fixed effect also transform the covariates (e.g., taking first differences can result in the estimating equation only including the change in covariates over time, not their level, and also drop time-invariant covariates altogether). We provide simple diagnostics for assessing how susceptible a TWFE regression is to hidden linearity bias based on reformulating the TWFE regression as a weighting estimator. Finally, we propose simple alternative estimation strategies that can circumvent these issues.

Motivation & Objective

• Evaluate how conditional parallel trends with time-varying covariates affect DID identification.
• Identify weaknesses of standard two-way fixed effects (TWFE) regressions in this setting.
• Develop diagnostics to assess misspecification bias from TWFE.
• Propose alternative estimation strategies that are robust to such misspecification.

Proposed method

• Formalize the conditional parallel trends framework with time-varying covariates and time-invariant covariates.
• Show that TWFE can be biased due to linearity violations, covariate level effects, and time-invariant covariates.
• Decompose the TWFE coefficient into weighted averages of conditional effects using linear projections (Propositions 2 and 3).
• Express the TWFE estimate as a weighted average of treated versus untreated potential outcome paths (Theorem 1).
• Provide regression adjustment strategies that accommodate time-varying covariates.
• Introduce a doubly robust estimand for the ATT and discuss machine learning implementations.

Experimental results

Research questions

• RQ1 Under a conditional parallel trends assumption with covariates, what is the target estimand for DID?
• RQ2 When does the TWFE coefficient fail to identify the ATT or a meaningful causal parameter under covariate conditioning?
• RQ3 How can researchers diagnose hidden linearity bias in TWFE regressions with time-varying covariates?
• RQ4 What alternative estimation strategies can reliably recover causal effects when TWFE is misspecified (e.g., regression adjustment, doubly robust methods)?

Key findings

• TWFE regressions can be non-robust to conditional parallel trends even with two periods due to (i) linearity violations, (ii) dependence on covariate levels, and (iii) time-invariant covariates.
• TWFE estimates may exhibit negative weights and weight-reversal even when heterogeneity is present, complicating causal interpretation.
• The authors provide diagnostics that recast the TWFE coefficient as implicit regression weights to assess balance of covariates under weighting.
• They propose regression adjustment that allows covariates to affect the path of untreated outcomes, and a doubly robust ATT estimand.
• The paper discusses asymptotic interpretation and practical estimation issues, including high-dimensional covariate spaces and potential machine learning implementations.