[Paper Review] Generalization of Change-Point Detection in Time Series Data Based on Direct Density Ratio Estimation

M. Hushchyn, A. Ustyuzhanin|arXiv (Cornell University)|Jan 17, 2020

Statistical Methods and Inference18 references27 citations

TL;DR

This paper proposes a generalization of change-point detection in time series using direct density ratio estimation with modern machine learning models, including Gradient Boosting Trees and Neural Networks. By replacing traditional kernel-based methods like RuLSIF with flexible classifiers and regressors, the approach achieves superior performance on synthetic and real-world datasets, particularly in high-dimensional or noisy settings where kernel methods struggle.

ABSTRACT

The goal of the change-point detection is to discover changes of time series distribution. One of the state of the art approaches of the change-point detection are based on direct density ratio estimation. In this work we show how existing algorithms can be generalized using various binary classification and regression models. In particular, we show that the Gradient Boosting over Decision Trees and Neural Networks can be used for this purpose. The algorithms are tested on several synthetic and real-world datasets. The results show that the proposed methods outperform classical RuLSIF algorithm. Discussion of cases where the proposed algorithms have advantages over existing methods are also provided.

Motivation & Objective

• To extend state-of-the-art change-point detection beyond kernel-based methods like RuLSIF by leveraging modern machine learning models.
• To address limitations of kernel methods in high-dimensional or noisy time series where distance-based estimation fails.
• To improve detection accuracy and robustness by using binary classifiers and regression models to estimate density ratios directly.
• To evaluate the performance of these generalized models on both synthetic and real-world time series datasets.
• To provide a framework for using flexible models such as GBDT and neural networks in direct density ratio estimation for change-point detection.

Proposed method

• Uses direct density ratio estimation via binary classification models (e.g., GBDT, neural networks) to estimate the ratio of test and reference distribution densities.
• Applies the RuLSIF loss function in a regression setting to train models for density ratio estimation without explicit density estimation.
• Employs a sliding window approach to form reference and test samples from time series sequences, capturing temporal dependencies.
• Estimates a dissimilarity score based on the predicted density ratio, which peaks at the change-point location.
• Uses ROC AUC as the primary evaluation metric, with labels assigned to time windows around the true change-point.
• Trains models using a modified loss function that minimizes the weighted squared error between predicted and true density ratios.

Experimental results

Research questions

• RQ1 Can modern machine learning models like GBDT and neural networks outperform classical kernel-based methods (e.g., RuLSIF) in change-point detection?
• RQ2 In what scenarios do non-kernel models provide a significant advantage over kernel-based density ratio estimation?
• RQ3 How do GBDT and neural networks handle noisy or high-dimensional time series components that degrade kernel-based methods?
• RQ4 Does direct density ratio estimation using flexible models improve detection accuracy and reduce delay in identifying change-points?
• RQ5 How does the performance of these models vary across synthetic and real-world time series with different change-point characteristics?

Key findings

• The GBDT-based and neural network-based models consistently outperform the classical RuLSIF algorithm across all synthetic datasets, with AUC improvements of up to 9.7 percentage points on Dataset 1.
• On the Kepler exoplanet light curve dataset, the GBDT-based model achieved an AUC of 0.950, compared to 0.844 for RuLSIF, demonstrating superior detection in real-world astrophysical signals.
• On the IRIS seismogram dataset, the GBDT model achieved an AUC of 0.980, significantly outperforming RuLSIF (0.971), indicating robustness in geophysical signal detection.
• The proposed models are less sensitive to uninformative noise components in time series, as evidenced by their consistent performance gains over RuLSIF in noisy synthetic datasets.
• Neural networks and GBDT models show better generalization in high-dimensional settings where kernel-based methods fail due to the curse of dimensionality.
• The use of direct density ratio estimation with flexible models enables detection of complex, non-linear distribution shifts that kernel methods may miss.