[Paper Review] Distribution-Free Detection of Structured Anomalies: Permutation and Rank-Based Scans

Ery Arias-Castro, Rui M. Castro+2 more|arXiv (Cornell University)|Aug 12, 2015

Data-Driven Disease Surveillance38 references3 citations

TL;DR

This paper proposes two distribution-free methods—permutation-based and rank-based scan statistics—for detecting structured anomalies when the null distribution is unknown. It demonstrates that both methods incur only a negligible power loss compared to an oracle scan test under natural exponential families, including normal and Poisson models, with the rank scan offering computational advantages and robustness to outliers.

ABSTRACT

The scan statistic is by far the most popular method for anomaly detection, being popular in syndromic surveillance, signal and image processing, and target detection based on sensor networks, among other applications. The use of the scan statistics in such settings yields a hypothesis testing procedure, where the null hypothesis corresponds to the absence of anomalous behavior. If the null distribution is known, then calibration of a scan-based test is relatively easy, as it can be done by Monte Carlo simulation. When the null distribution is unknown, it is less straightforward. We investigate two procedures. The first one is a calibration by permutation and the other is a rank-based scan test, which is distribution-free and less sensitive to outliers. Furthermore, the rank scan test requires only a one-time calibration for a given data size making it computationally much more appealing. In both cases, we quantify the performance loss with respect to an oracle scan test that knows the null distribution. We show that using one of these calibration procedures results in only a very small loss of power in the context of a natural exponential family. This includes the classical normal location model, popular in signal processing, and the Poisson model, popular in syndromic surveillance. We perform numerical experiments on simulated data further supporting our theory and also on a real dataset from genomics.

Motivation & Objective

• To address the challenge of calibrating scan statistics when the null distribution is unknown, which is common in real-world applications like syndromic surveillance and sensor networks.
• To develop distribution-free alternatives to traditional scan tests that do not rely on parametric assumptions about the underlying data distribution.
• To quantify the performance loss of these nonparametric methods relative to an oracle scan test that knows the true null distribution.
• To demonstrate the computational and robustness advantages of the rank-based scan test, especially in high-dimensional or outlier-prone settings.
• To validate theoretical findings through numerical experiments on simulated data and a real genomics dataset.

Proposed method

• Uses permutation-based calibration by resampling data under the null to estimate the null distribution of the scan statistic, enabling valid hypothesis testing without parametric assumptions.
• Proposes a rank-based scan test that replaces original observations with their ranks before computing the scan statistic, ensuring distribution-free inference.
• Applies the scan statistic to all contiguous intervals (or regions) of interest, computing the sum of values (or ranks) over each interval and identifying the maximum as the test statistic.
• Derives asymptotic properties of the scan statistic under the null and alternative hypotheses using extreme value theory and concentration inequalities.
• Establishes theoretical bounds on the p-value of the rank scan test, showing it decays to zero under the alternative, ensuring consistency.
• Uses a one-time calibration for the rank scan test based on data size, making it computationally efficient compared to repeated Monte Carlo simulations.

Experimental results

Research questions

• RQ1 How can scan statistics be calibrated when the null distribution is unknown, particularly in settings like syndromic surveillance or sensor networks?
• RQ2 What is the performance loss of permutation-based and rank-based scan tests compared to an oracle scan test that knows the null distribution?
• RQ3 Does the rank-based scan test offer advantages in robustness to outliers and computational efficiency over permutation-based calibration?
• RQ4 Under what conditions does the rank-based scan test maintain high statistical power in the presence of unknown null distributions?
• RQ5 Can the proposed methods be theoretically justified and empirically validated in both simulated and real-world data, such as genomics data?

Key findings

• The permutation-based scan test achieves power close to the oracle scan test, with only a small loss in detection power under the normal and Poisson models.
• The rank-based scan test is distribution-free and exhibits minimal power loss compared to the oracle test, especially under the normal location and Poisson models.
• The rank scan test requires only a one-time calibration based on data size, making it significantly more computationally efficient than permutation-based methods.
• Theoretical analysis shows that under the alternative hypothesis, the p-value of the rank scan test decays to zero at an exponential rate, confirming consistency.
• Numerical experiments on simulated data confirm the theoretical findings, showing robust performance under various anomaly configurations.
• Empirical validation on a real genomics dataset demonstrates the practical utility and robustness of the rank-based scan test in real-world anomaly detection.