[Paper Review] Donut Regression Discontinuity Designs

Cladia Noack, Chistoph Rothe|arXiv (Cornell University)|Aug 28, 2023

Advanced Causal Inference Techniques4 citations

TL;DR

This paper provides a formal econometric framework for donut regression discontinuity (RD) designs, showing that such estimators suffer from higher bias and variance than conventional RD estimators under standard assumptions. It introduces valid bias-aware inference, including confidence intervals and specification tests, demonstrating that donut RD can detect model misspecification but at the cost of efficiency and precision.

ABSTRACT

We study the econometric properties of so-called donut regression discontinuity (RD) designs, a robustness exercise which involves repeating estimation and inference without the data points in some area around the treatment threshold. This approach is often motivated by concerns that possible systematic sorting of units, or similar data issues, in some neighborhood of the treatment threshold might distort estimation and inference of RD treatment effects. We show that donut RD estimators can have substantially larger bias and variance than contentional RD estimators, and that the corresponding confidence intervals can be substantially longer. We also provide a formal testing framework for comparing donut and conventional RD estimation results.

Motivation & Objective

• To address the lack of statistical theory supporting the widely used but heuristic 'donut' RD approach, which excludes data near the treatment threshold to test robustness.
• To formally analyze the bias, variance, and efficiency trade-offs of donut RD estimators compared to conventional RD estimators under standard RD assumptions.
• To develop valid statistical inference procedures—specifically, bias-aware confidence intervals and specification tests—for donut RD designs that account for the dependence between conventional and donut estimates.
• To evaluate the power and size properties of new specification tests for detecting violations of RD assumptions in the donut framework.
• To provide empirical guidance on when and why donut RD may be preferable or misleading, based on theoretical and simulation evidence.

Proposed method

• Derives asymptotic bias and variance expansions for donut RD estimators under local linear regression with triangular and uniform kernels, showing increases of 41–63% in bias and 53–61% in variance when excluding data within 10% of the bandwidth.
• Adapts bias-aware confidence intervals (Armstrong and Kolesár, 2018; Kolesár and Rothe, 2018) to the donut RD setting, proving they remain valid without adjustment, though their length increases by 22–28%.
• Proposes a new specification test based on the difference in estimators, $ \widehat{\Gamma} $, which has better local power than the conventional $ \widehat{\Delta} $-based test under alternative hypotheses.
• Uses a small donut asymptotic framework where the exclusion window shrinks as sample size grows, enabling theoretical analysis of bias and variance under the donut design.
• Employs simulation studies with $ n = 1,000 $, triangular kernel, bandwidth $ h = 0.49 $, and exclusion window $ d = 0.1 $, comparing conventional and donut RD under varying levels of conditional expectation deviation $ L \in \{0,10,\dots,40\} $.
• Applies the proposed inference tools to a real-world example: infant mortality near the 1500g birthweight threshold, showing how donut RD can alter conclusions but requires formal statistical validation.

Experimental results

Research questions

• RQ1 How do the bias and variance of donut RD estimators compare to those of conventional RD estimators under standard RD assumptions?
• RQ2 Are bias-aware confidence intervals valid in the donut RD setting, and how do their lengths compare to conventional intervals?
• RQ3 Can valid statistical tests be constructed to compare conventional and donut RD estimands, accounting for their high correlation?
• RQ4 What is the power of the proposed specification test in detecting model misspecification in donut RD designs compared to existing approaches?
• RQ5 Under what conditions does donut RD estimation yield more accurate estimates than conventional RD, based on empirical and simulation evidence?

Key findings

• Donut RD estimators exhibit 41–63% higher bias and 53–61% higher variance than conventional RD estimators when excluding data within 10% of the bandwidth, under standard RD assumptions.
• Bias-aware confidence intervals remain valid in the donut RD setting, but their asymptotic length increases by 22–28% due to reduced effective sample size and increased variance.
• The proposed specification test based on $ \widehat{\Gamma} $ has strictly greater power than the conventional $ \widehat{\Delta} $-based test in the simulation setup, particularly under alternatives with strong deviations near the cutoff.
• Empirical coverage of donut RD confidence intervals remains correct across all simulation scenarios, while conventional confidence intervals deteriorate in coverage as the true conditional mean departs from smoothness near the threshold.
• The root mean squared error (RMSE) of the donut estimator only becomes smaller than that of the conventional estimator under extreme deviations from RD assumptions (e.g., $ L = 40 $), indicating that donut RD is not generally more efficient.
• In the empirical application to infant mortality at the 1500g threshold, the conventional RD estimate was 0.95% with a standard error of 0.22%, while the donut RD estimate dropped to 0.16% with a standard error of 0.28%, illustrating the need for formal inference to assess whether the difference is statistically meaningful.