[Paper Review] Consistent detection and optimal localization of all detectable change points in piecewise stationary arbitrarily sparse network-sequences

Sharmodeep Bhattacharyya, Shirshendu Chatterjee+1 more|arXiv (Cornell University)|Sep 4, 2020

Data-Driven Disease Surveillance40 references4 citations

TL;DR

This paper proposes a consistent offline change point detection and localization method for piecewise stationary, arbitrarily sparse network-sequences using adaptively trimmed CUSUM statistics on adjacency matrices. It achieves consistent detection and optimal localization of all detectable change points without requiring prior bounds on sparsity, number of change points, or minimum separation, under minimal asymptotic conditions on network size or average degree.

ABSTRACT

We consider the offline change point detection and localization problem in the context of piecewise stationary networks, where the observable is a finite sequence of networks. We develop algorithms involving some suitably modified CUSUM statistics based on adaptively trimmed adjacency matrices of the observed networks for both detection and localization of single or multiple change points present in the input data. We provide rigorous theoretical analysis and finite sample estimates evaluating the performance of the proposed methods when the input (finite sequence of networks) is generated from an inhomogeneous random graph model, where the change points are characterized by the change in the mean adjacency matrix. We show that the proposed algorithms can detect (resp. localize) all change points, where the change in the expected adjacency matrix is above the minimax detectability (resp. localizability) threshold, consistently without any a priori assumption about (a) a lower bound for the sparsity of the underlying networks, (b) an upper bound for the number of change points, and (c) a lower bound for the separation between successive change points, provided either the minimum separation between successive pairs of change points or the average degree of the underlying networks goes to infinity arbitrarily slowly. We also prove that the above condition is necessary to have consistency.

Motivation & Objective

• Address the challenge of detecting and localizing change points in time-series of sparse, piecewise stationary networks without prior assumptions on sparsity, number of change points, or minimum separation.
• Develop a theoretically grounded method that ensures consistent detection and optimal localization of all detectable change points in network-sequences.
• Provide finite-sample performance guarantees under an inhomogeneous random graph model where changes are defined by shifts in the mean adjacency matrix.
• Establish necessary and sufficient conditions for consistency, showing that the proposed conditions are tight in terms of asymptotic behavior of average degree or inter-change-point separation.
• Demonstrate superiority over existing methods via empirical evaluation on simulated data sets.

Proposed method

• Adapt the classical CUSUM statistic by applying adaptive trimming to adjacency matrices to reduce noise and improve sensitivity to structural changes.
• Use a two-stage algorithm: first detect change points via a modified CUSUM scan over sliding windows, then refine localization using interval-based estimation.
• Introduce cushion and interval lengths (κ and Λ) to control detection sensitivity and localization precision, with Λ ≤ κ for stability.
• Define signal strength S(Q) as a key quantity that determines localization accuracy, with higher values enabling finer localization.
• Employ concentration inequalities and spectral norms to bound estimation error and derive high-probability bounds on localization deviation.
• Apply Wild Binary Segmentation in Algorithm 2 to recursively detect multiple change points, with theoretical guarantees on recovery of all K change points with high probability.

Experimental results

Research questions

• RQ1 Can change point detection and localization be consistently performed in arbitrarily sparse, piecewise stationary network-sequences without prior knowledge of sparsity or change-point separation?
• RQ2 What is the minimal signal strength required for consistent detection and localization of change points in such network-sequences?
• RQ3 How does the performance of the proposed method scale with network size, average degree, and inter-change-point distance?
• RQ4 Is the proposed condition on asymptotic growth of average degree or minimum separation necessary for consistency?
• RQ5 Can the method achieve optimal localization error without assuming a lower bound on the number of nodes or network density?

Key findings

• The proposed method achieves consistent detection of all change points when the signal strength S(Q) exceeds a threshold proportional to √(d / (Λ ∧ κ)) × √(ζ + log(|L|)/log(n)), ensuring high-probability detection.
• Localization error is bounded by O(Λ × (C₁ + c₁Ψμ) / S(Q)) × √(d / (Λ ∧ κ)) with high probability, improving as signal strength increases.
• When the minimum separation between change points or average degree grows arbitrarily slowly to infinity, the method achieves consistency without any a priori assumptions.
• The condition κ > g(Q) and S(Q) ≥ (C₁ + c₁Ψμ) × √(d / (κ) × (ζ + log(M)/log(n))) is both sufficient and necessary for consistency in multiple change point detection.
• Empirical evaluation shows superior performance over existing methods in terms of detection accuracy and localization precision on simulated sparse network-sequences.
• The method achieves exact recovery of all K change points with high probability under the stated conditions, even when the number of change points is unknown and unbounded.