[Paper Review] Difference-in-Differences with Compositional Changes

Pedro H. C. Sant’Anna, Qi Xu|arXiv (Cornell University)|Apr 27, 2023

Advanced Causal Inference Techniques4 citations

TL;DR

This paper develops a doubly robust difference-in-differences estimator that accounts for compositional changes in repeated cross-sectional data, using nonparametric estimation of outcome regressions and generalized propensity scores with local polynomial methods. It establishes semiparametric efficiency, derives a Hausman-type test for compositional changes, and shows that ignoring such changes leads to bias and efficiency loss, with empirical validation in a corruption and tariff liberalization study.

ABSTRACT

This paper studies Difference-in-Differences (DiD) setups with repeated cross-sectional data and potential compositional changes across time periods. We begin our analysis by deriving the efficient influence function and the semiparametric efficiency bound for the average treatment effect on the treated (ATT). We introduce nonparametric estimators that attain the semiparametric efficiency bound under mild rate conditions on the estimators of the nuisance functions, exhibiting a type of rate doubly robust (DR) property. Additionally, we document a trade-off related to compositional changes: We derive the asymptotic bias of DR DiD estimators that erroneously exclude compositional changes and the efficiency loss when one fails to correctly rule out compositional changes. We propose a nonparametric Hausman-type test for compositional changes based on these trade-offs. The finite sample performance of the proposed DiD tools is evaluated through Monte Carlo experiments and an empirical application. We consider extensions of our framework that accommodate double machine learning procedures with cross-fitting, and setups when some units are observed in both pre- and post-treatment periods. As a by-product of our analysis, we present a new uniform stochastic expansion of the local polynomial multinomial logit estimator, which may be of independent interest.

Motivation & Objective

• To address the limitations of existing DiD methods that assume no compositional changes across time periods, which can lead to biased estimates in practice.
• To derive the semiparametric efficiency bound and efficient influence function for the ATT under compositional changes, without relying on the no-compositional-change assumption.
• To propose a nonparametric, rate doubly-robust DiD estimator that achieves the efficiency bound under mild smoothness conditions on nuisance function estimators.
• To develop a nonparametric Hausman-type test to assess the validity of the no-compositional-change assumption.
• To evaluate finite-sample performance via Monte Carlo simulations and an empirical application on tariff liberalization and corruption.

Proposed method

• Derives the efficient influence function and semiparametric efficiency bound for the ATT in DiD with compositional changes, under conditional parallel trends.
• Proposes a generic nonparametric estimator based on the efficient influence function, achieving the efficiency bound when nuisance functions are estimated with sufficient rate of convergence.
• Uses local polynomial regression for outcome regression models and local multinomial logit for generalized propensity score estimation, with data-driven bandwidth selection via cross-validation.
• Introduces a novel uniform stochastic expansion for local multinomial logit estimators with mixed discrete and continuous covariates, a contribution of independent interest.
• Develops a nonparametric Hausman-type test comparing estimators under and above the no-compositional-change assumption to test its plausibility.
• Applies the method to a real-world dataset on tariff liberalization and bribery payments, comparing DR DiD estimators with and without compositional change robustness.

Experimental results

Research questions

• RQ1 How does the presence of compositional changes across time periods affect the identification and estimation of the average treatment effect on the treated (ATT) in DiD frameworks?
• RQ2 What is the semiparametric efficiency bound for the ATT when compositional changes are allowed, and how can it be achieved via nonparametric estimation?
• RQ3 What are the finite-sample consequences of incorrectly excluding compositional changes in DR DiD estimators, and how does this affect bias and efficiency?
• RQ4 Can a nonparametric Hausman-type test reliably detect the presence of compositional changes in repeated cross-sectional DiD data?
• RQ5 How do the proposed robust estimators compare to existing DR DiD methods—particularly Sant2020doubly—when compositional changes are present?

Key findings

• The proposed doubly robust DiD estimator achieves the semiparametric efficiency bound under mild smoothness conditions on nuisance function estimators, even when compositional changes are present.
• The estimand proposed by Sant2020doubly is not doubly robust in the presence of compositional changes, even when all nuisance functions are correctly specified, and fails to identify the ATT.
• Ignoring compositional changes leads to asymptotic bias in DR DiD estimators, and failing to account for them results in a measurable efficiency loss compared to the no-compositional-change benchmark.
• The Monte Carlo simulations confirm that the proposed estimator maintains good finite-sample performance and that the Hausman-type test has good size and power in detecting compositional changes.
• In the empirical application on tariff liberalization and bribery, the proposed robust DR DiD estimator estimates a 28–43 percentage point reduction in bribery probability, with larger effects than TWFE, and the Hausman test fails to reject the no-compositional-change assumption when clustered standard errors are used.
• The results suggest that while compositional changes are not the primary concern in this application, the proposed method provides a more reliable inference tool when such changes are suspected.