[Paper Review] Estimation of Policy-Relevant Causal Effects in the Presence of Interference with an Application to the Philadelphia Beverage Tax

Gary Hettinger, Christina A. Roberto+2 more|arXiv (Cornell University)|Jan 17, 2023

Spatial and Panel Data AnalysisEconomics, Econometrics and Finance3 citations

TL;DR

This paper proposes a doubly robust difference-in-differences framework that accounts for interference due to cross-border shopping in policy evaluations, applying it to estimate the causal effects of Philadelphia's sugar-sweetened beverage tax on sales in Philadelphia and neighboring counties. The method improves estimation by adjusting for both confounding and spillover effects, revealing significant heterogeneity in treatment effects by geography and season, with pharmacies showing the largest declines in sales.

ABSTRACT

To comprehensively evaluate a public policy intervention, researchers must consider the effects of the policy not just on the implementing region, but also nearby, indirectly-affected regions. For example, an excise tax on sweetened beverages in Philadelphia was shown to not only be associated with a decrease in volume sales of taxed beverages in Philadelphia, but also an increase in sales in bordering counties not subject to the tax. The latter association may be explained by cross-border shopping behaviors of Philadelphia residents and indicate a causal effect of the tax on nearby regions, which may offset the total effect of the intervention. To estimate causal effects in this setting, we extend difference-in-differences methodology to account for such interference between regions and adjust for potential confounding present in quasi-experimental evaluations. Our doubly robust estimators for the average treatment effect on the treated and neighboring control relax standard assumptions on interference and model specification. We apply these methods to evaluate the change in volume sales of taxed beverages in 231 Philadelphia and bordering county stores due to the Philadelphia beverage tax. We also use our methods to explore the heterogeneity of effects across geographic features.

Motivation & Objective

• To address interference in policy evaluations where individuals bypass taxed regions by shopping in nearby untaxed areas, a phenomenon violating the Stable Unit Treatment Value Assumption (SUTVA).
• To estimate policy-relevant causal effects—specifically the average treatment effect on the treated (ATT) and average treatment effect on the neighbors (ATN)—in the presence of both confounding and interference.
• To develop and validate a method that relaxes strict model assumptions by using doubly robust estimators that require only one of the outcome or propensity score models to be correctly specified.
• To apply the method to real-world data on volume sales of taxed beverages in Philadelphia and bordering counties to assess the true impact of the Philadelphia Beverage Tax.
• To explore heterogeneity in treatment effects across geographic proximity and seasonal variation, offering new insights for policy evaluation.

Proposed method

• Extends the traditional difference-in-differences (DiD) framework to incorporate exposure mappings that define which units are affected by the policy and which are indirectly influenced by cross-border behavior.
• Uses doubly robust estimators combining outcome regression and inverse probability weighting (IPW) to estimate ATT and ATN, ensuring consistency if either the outcome model or the propensity score model is correctly specified.
• Applies exposure mapping to define treatment status not only for Philadelphia but also for bordering counties, capturing indirect effects due to bypass behavior.
• Employs a two-way fixed effects (TWFE) model with adjustments for time-varying confounders and uses bootstrap-based confidence intervals to improve coverage in finite samples.
• Uses simulation studies to evaluate performance under various model misspecification scenarios, comparing standard DiD, outcome regression (OR), IPW, and doubly robust (DR) estimators.
• Incorporates geographic and seasonal covariates to explore heterogeneity in treatment effects, identifying subpopulations with stronger or weaker responses.

Experimental results

Research questions

• RQ1 What is the true causal effect of the Philadelphia Beverage Tax on volume sales in Philadelphia when accounting for cross-border shopping behavior?
• RQ2 How do the effects of the tax differ between Philadelphia and neighboring counties due to interference from residents bypassing the tax?
• RQ3 To what extent do geographic proximity and seasonal variation moderate the treatment effect of the beverage tax on sales?
• RQ4 How do doubly robust estimators improve the reliability of causal inference in the presence of both confounding and interference compared to standard DiD methods?
• RQ5 What is the magnitude of the 'bypass effect'—whereby residents of Philadelphia shift purchases to nearby untaxed counties—and how does it affect the net policy impact?

Key findings

• The doubly robust DiD estimator significantly reduces bias compared to standard TWFE when time-varying confounding and interference are present, particularly in scenarios with model misspecification.
• The study estimates that the Philadelphia Beverage Tax led to a 51% decline in volume sales of taxed beverages in Philadelphia in the first year post-tax, consistent with prior DiD studies.
• A substantial portion of the policy’s effect—estimated at 25–30%—was offset by cross-border shopping, indicating a meaningful bypass effect in neighboring counties.
• Pharmacies showed the largest treatment effects, with significantly greater declines in sales compared to other store types, suggesting higher sensitivity to price changes.
• Treatment effects were heterogeneous by season, with larger declines observed in winter months, possibly due to changes in consumer behavior or inventory patterns.
• The method revealed that the ATN (average treatment effect on neighbors) was positive and significant in bordering counties, indicating increased sales due to cross-border shopping, which partially offsets the intended policy impact.