[Paper Review] Permutation-based tests for discontinuities in event studies

Federico A. Bugni, Jia Li+1 more|arXiv (Cornell University)|Jul 20, 2020

Financial Risk and Volatility Modeling29 references4 citations

TL;DR

This paper proposes a permutation-based test to detect discontinuities in event studies using time-series data, where the test statistic measures the distance between empirical distribution functions on either side of a known cutoff. The method is asymptotically valid under a high-level condition of conditional independence and maintains finite-sample validity even with dependent data, offering a robust tool for detecting jumps in economic variables like volatility and liquidity.

ABSTRACT

We propose using a permutation test to detect discontinuities in an underlying economic model at a known cutoff point. Relative to the existing literature, we show that this test is well suited for event studies based on time-series data. The test statistic measures the distance between the empirical distribution functions of observed data in two local subsamples on the two sides of the cutoff. Critical values are computed via a standard permutation algorithm. Under a high-level condition that the observed data can be coupled by a collection of conditionally independent variables, we establish the asymptotic validity of the permutation test, allowing the sizes of the local subsamples to be either be fixed or grow to infinity. In the latter case, we also establish that the permutation test is consistent. We demonstrate that our high-level condition can be verified in a broad range of problems in the infill asymptotic time-series setting, which justifies using the permutation test to detect jumps in economic variables such as volatility, trading activity, and liquidity. These potential applications are illustrated in an empirical case study for selected FOMC announcements during the ongoing COVID-19 pandemic.

Motivation & Objective

• To develop a permutation test for detecting discontinuities in economic time-series data at a known cutoff point.
• To establish asymptotic validity of the test under a high-level condition of conditional independence of observed data given a sigma-algebra.
• To extend the applicability of permutation tests beyond i.i.d. assumptions to settings with dependent, nonstationary, or persistent processes common in financial and macroeconomic event studies.
• To verify the high-level condition in infill asymptotic settings, particularly for high-frequency financial data.
• To demonstrate the method’s empirical relevance through an application to FOMC announcements during the COVID-19 pandemic.

Proposed method

• The test uses a Cramér-von Mises-type test statistic measuring the L2 distance between empirical distribution functions from two local subsamples on either side of the cutoff.
• Critical values are computed via a standard permutation algorithm that resamples the data while preserving the conditional dependence structure.
• The method relies on a coupling construction to couple the observed data with i.i.d. copies under the null, ensuring asymptotic validity under conditional independence.
• The test is extended to a broad class of rank-based test statistics satisfying Assumption 5.1, including Kolmogorov-Smirnov and Cramér-von Mises statistics.
• Theoretical validity is established under a high-level condition that allows for unrestricted persistence and nonstationarity in the underlying process.
• Consistency is proven when the sample size of the local subsamples grows to infinity and the test statistic diverges under the alternative.

Experimental results

Research questions

• RQ1 Can permutation tests be used to detect discontinuities in event studies with dependent time-series data?
• RQ2 Under what conditions is a permutation test asymptotically valid when the data are not i.i.d. but conditionally independent?
• RQ3 Can the proposed method detect jumps in economic variables such as volatility, trading activity, and liquidity in high-frequency settings?
• RQ4 How does the test perform when the local subsample sizes grow to infinity versus remain fixed?
• RQ5 Is the test robust to nonstationarity and long-memory dependence in the underlying process?

Key findings

• The permutation test is asymptotically valid under a high-level condition that the observed data can be coupled by conditionally independent variables, even when the subsample sizes are fixed.
• When the local subsample sizes grow to infinity, the test is consistent, meaning its power approaches one under the alternative hypothesis.
• The test maintains finite-sample validity under the null when the data are exchangeable under the null, which holds under the conditional independence assumption.
• The high-level condition is verified in a broad class of infill asymptotic time-series models, justifying the use of the test for detecting jumps in volatility and liquidity.
• The method applies to a wide class of rank-based test statistics, including Cramér-von Mises and Kolmogorov-Smirnov, due to the generalization via Assumption 5.1.
• An empirical application to FOMC announcements during the COVID-19 pandemic demonstrates the test's ability to detect significant discontinuities in financial market variables.