[Paper Review] Data Smashing 2.0: Sequence Likelihood (SL) Divergence For Fast Time Series Comparison

Yi Huang, Ishanu Chattopadhyay|arXiv (Cornell University)|Sep 26, 2019

Time Series Analysis and Forecasting11 references4 citations

TL;DR

This paper introduces Smash2.0, a universal, fast, and feature-engineering-free metric for time series comparison based on Sequence Likelihood (SL) divergence derived from Probabilistic Finite-State Automata (PFSA). It enables efficient estimation of model-level similarity between time series by measuring divergence in their underlying stochastic generators, significantly outperforming prior methods like Dynamic Time Warping and original Data Smashing in speed and discrimination accuracy on synthetic and real-world EEG and gait data.

ABSTRACT

Recognizing subtle historical patterns is central to modeling and forecasting problems in time series analysis. Here we introduce and develop a new approach to quantify deviations in the underlying hidden generators of observed data streams, resulting in a new efficiently computable universal metric for time series. The proposed metric is in the sense that we can compare and contrast data streams regardless of where and how they are generated and without any feature engineering step. The approach proposed in this paper is conceptually distinct from our previous work on data smashing, and vastly improves discrimination performance and computing speed. The core idea here is the generalization of the notion of KL divergence often used to compare probability distributions to a notion of divergence in time series. We call this the sequence likelihood (SL) divergence, which may be used to measure deviations within a well-defined class of discrete-valued stochastic processes. We devise efficient estimators of SL divergence from finite sample paths and subsequently formulate a universal metric useful for computing distance between time series produced by hidden stochastic generators.

Motivation & Objective

• To develop a universal, computationally efficient metric for comparing time series without requiring domain-specific feature engineering.
• To address the limitation of existing methods in distinguishing time series generated by the same stochastic process despite low pointwise similarity.
• To generalize KL divergence to stochastic processes via a new divergence measure—Sequence Likelihood (SL) divergence—enabling model-level comparison.
• To enable robust classification and clustering of time series from diverse sources, such as EEG and gait data, by estimating similarity in their underlying generative processes.
• To provide a scalable framework for time series analysis that operates directly on raw data streams using quantization and PFSA inference.

Proposed method

• The method models time series as realizations of discrete-valued stochastic processes using Probabilistic Finite-State Automata (PFSA), which can represent non-Markovian dynamics.
• It defines Sequence Likelihood (SL) divergence as a generalization of KL divergence to stochastic processes, quantifying deviation between their generating models.
• The approach uses log-likelihood convergence of finite sample paths to estimate SL divergence efficiently, avoiding costly dimensionality reduction.
• A quantization scheme is applied to continuous time series data to map them into finite alphabets suitable for PFSA modeling.
• An inference algorithm, GenESeSS, is used to learn the PFSA model from observed sequences, enabling estimation of entropy rate and KL divergence in closed form.
• The final metric, Smash2.0, computes pairwise distances between time series based on SL divergence, forming a universal distance matrix for downstream tasks.

Experimental results

Research questions

• RQ1 Can a universal metric be developed to compare time series without requiring feature engineering or domain-specific knowledge?
• RQ2 How can KL divergence be generalized to stochastic processes to measure model-level differences rather than sample-path similarity?
• RQ3 Can SL divergence be efficiently estimated from finite-length time series to enable fast and accurate time series comparison?
• RQ4 How does the performance of the new metric compare to established methods like DTW and original Data Smashing in terms of speed and discrimination accuracy?
• RQ5 Can the method effectively disambiguate subtle differences in complex real-world time series, such as EEG patterns during rest vs. movement or gait patterns across individuals?

Key findings

• Smash2.0 outperforms both original Data Smashing and Dynamic Time Warping (DTW) in discrimination accuracy on synthetic time series, with significantly improved computational efficiency.
• On EEG data, Smash2.0 successfully disambiguates brain state differences between rest and (imaginary) movement in subject S004, with a correlation coefficient r(D) of .572 for channel 21, while showing minimal differences in S001, consistent with clinical expectations.
• For gait recognition, the method identifies distinct motion patterns across individuals: in the z-direction, the distance matrix for the first two participants shows a clear distinction with r(D) = .340, enabling reliable user identification.
• The method achieves high performance in user identification using multi-channel accelerometer data, where combining directional measurements with channel-specific quantization schemes enhances classification accuracy beyond single-channel approaches.
• The use of PFSA enables closed-form computation of entropy rate and KL divergence, supporting fast and stable estimation of SL divergence from finite sample paths.
• The framework demonstrates robustness across diverse data types, including irregularly sampled and non-stationary time series, by modeling the underlying stochastic generators rather than raw signal features.