[Paper Review] First-order optimal sequential subspace change-point detection

Liyan Xie, George V. Moustakides+1 more|arXiv (Cornell University)|Jun 28, 2018

Advanced Statistical Process Monitoring8 references4 citations

TL;DR

This paper proposes the Subspace-CUSUM procedure for sequential detection of change-points in high-dimensional streaming data where changes occur in a low-rank subspace structure. By combining CUSUM statistics with adaptive subspace estimation and optimal parameter tuning, the method achieves first-order asymptotic optimality, meaning its expected detection delay converges to that of the ideal CUSUM detector with full knowledge of pre- and post-change parameters as the average run length increases.

ABSTRACT

We consider the sequential change-point detection problem of detecting changes that are characterized by a subspace structure. Such changes are frequent in high-dimensional streaming data altering the form of the corresponding covariance matrix. In this work we present a Subspace-CUSUM procedure and demonstrate its first-order asymptotic optimality properties for the case where the subspace structure is unknown and needs to be simultaneously estimated. To achieve this goal we develop a suitable analytical methodology that includes a proper parameter optimization for the proposed detection scheme. Numerical simulations corroborate our theoretical findings.

Motivation & Objective

• To address the challenge of detecting structured changes in high-dimensional streaming data where the covariance matrix shifts due to a low-rank subspace change.
• To develop a sequential detection method that is robust to unknown subspace directions and signal strength.
• To achieve first-order asymptotic optimality by minimizing expected detection delay relative to the ideal CUSUM detector with full knowledge of change parameters.
• To provide a practical, on-line detection scheme that balances detection speed and false alarm control through optimized window size and drift parameter.

Proposed method

• Proposes the Subspace-CUSUM procedure, which applies a CUSUM statistic to projections of data onto estimated subspaces to detect changes in covariance structure.
• Uses a sliding window of size $ w $ to estimate the subspace and compute a cumulative sum of residuals, with a drift parameter $ d $ to control sensitivity.
• Optimizes the window size $ w $ and drift $ d $ as functions of the average run length $ \gamma $, using asymptotic analysis to ensure optimality.
• Applies a transformation to reduce the switching subspace problem to an emerging subspace problem via orthogonal projection, preserving detection performance while simplifying analysis.
• Derives the expected detection delay (EDD) using large-sample approximations and Laplace's method, leading to an analytical expression for EDD in terms of $ \gamma $, $ \rho $, and $ k $.
• Establishes first-order asymptotic optimality by showing the ratio of EDDs between Subspace-CUSUM and the ideal CUSUM tends to 1 as $ \gamma \to \infty $.

Experimental results

Research questions

• RQ1 Can a sequential change-point detection procedure be designed that is both effective for structured covariance changes and asymptotically optimal when the subspace is unknown?
• RQ2 How should the window size and drift parameter be selected to minimize expected detection delay under a fixed average run length constraint?
• RQ3 Does the Subspace-CUSUM procedure achieve first-order asymptotic optimality, meaning its detection delay ratio to the ideal CUSUM tends to 1 as the average run length increases?
• RQ4 How does the performance of Subspace-CUSUM compare to alternative methods such as the largest eigenvalue test and exact CUSUM in finite-sample settings?

Key findings

• The Subspace-CUSUM procedure achieves first-order asymptotic optimality: the ratio of its expected detection delay to that of the ideal CUSUM detector converges to 1 as the average run length $ \gamma \to \infty $.
• The optimal window size is $ w^* = \sqrt{\log \gamma} \cdot \frac{\sqrt{2(k-1)}}{\rho - \log(1+\rho)} (1+o(1)) $, which increases with $ \gamma $, ensuring asymptotic optimality.
• The expected detection delay of Subspace-CUSUM is $ \mathbb{E}_0[\mathcal{T}_{\rm C}] = \frac{2\log\gamma(1+o(1))}{(1+\rho)(1 - \frac{k-1}{w^*\rho}) - 1 - \log[(1+\rho)(1 - \frac{k-1}{w^*\rho})]} + w^* $, with the ratio to the ideal CUSUM approaching 1.
• Numerical simulations confirm that Subspace-CUSUM with optimized $ w $ significantly outperforms the largest eigenvalue method and approaches the performance of the ideal CUSUM.
• Even though the EDD ratio tends to 1, the absolute difference in EDD between Subspace-CUSUM and the ideal CUSUM grows as $ \Theta(\sqrt{\log \gamma}) $, indicating a non-vanishing but diminishing relative gap.