[Paper Review] Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data

David Hallac, Sagar Vare+2 more|arXiv (Cornell University)|Jun 10, 2017

Time Series Analysis and ForecastingComputer Science42 references19 citations

TL;DR

This paper proposes Toeplitz Inverse Covariance-based Clustering (TICC), a model-based method that simultaneously segments and clusters multivariate time series by learning sparse Markov random fields (MRFs) representing inter-sensor dependencies within each cluster. TICC uses dynamic programming for temporal segmentation and ADMM for estimating Toeplitz-structured inverse covariance matrices, enabling interpretable, structure-aware clustering that outperforms distance-based baselines on synthetic and real-world automotive sensor data.

ABSTRACT

Subsequence clustering of multivariate time series is a useful tool for discovering repeated patterns in temporal data. Once these patterns have been discovered, seemingly complicated datasets can be interpreted as a temporal sequence of only a small number of states, or clusters. For example, raw sensor data from a fitness-tracking application can be expressed as a timeline of a select few actions (i.e., walking, sitting, running). However, discovering these patterns is challenging because it requires simultaneous segmentation and clustering of the time series. Furthermore, interpreting the resulting clusters is difficult, especially when the data is high-dimensional. Here we propose a new method of model-based clustering, which we call Toeplitz Inverse Covariance-based Clustering (TICC). Each cluster in the TICC method is defined by a correlation network, or Markov random field (MRF), characterizing the interdependencies between different observations in a typical subsequence of that cluster. Based on this graphical representation, TICC simultaneously segments and clusters the time series data. We solve the TICC problem through alternating minimization, using a variation of the expectation maximization (EM) algorithm. We derive closed-form solutions to efficiently solve the two resulting subproblems in a scalable way, through dynamic programming and the alternating direction method of multipliers (ADMM), respectively. We validate our approach by comparing TICC to several state-of-the-art baselines in a series of synthetic experiments, and we then demonstrate on an automobile sensor dataset how TICC can be used to learn interpretable clusters in real-world scenarios.

Motivation & Objective

• To address the challenge of simultaneously segmenting and clustering high-dimensional multivariate time series into interpretable, recurring states.
• To improve interpretability beyond distance-based clustering by modeling conditional dependencies via Markov random fields (MRFs).
• To develop a scalable, model-based clustering framework that captures time-invariant correlation structures within subsequences.
• To enable discovery of meaningful, real-world behavioral patterns in sensor data, such as driving maneuvers in automotive datasets.
• To provide a method that clusters based on structural similarity rather than raw value alignment, making it robust to directional or symmetric variations (e.g., left vs. right turns).

Proposed method

• TICC models each cluster as a multilayer Markov random field (MRF) defined over a sliding window of fixed size, capturing time-invariant partial correlation structures.
• The MRFs are represented by sparse inverse covariance matrices (precision matrices), where zero entries indicate conditional independence between sensor pairs.
• The algorithm alternates between two steps: (1) assigning time points to clusters using dynamic programming to ensure temporal consistency, and (2) updating cluster-specific MRFs via ADMM to solve the sparse inverse covariance estimation problem.
• The inverse covariance matrices are constrained to be Toeplitz to enforce stationarity and reduce parameters, improving scalability and interpretability.
• The method uses an alternating minimization framework inspired by the EM algorithm, with closed-form updates for both subproblems.
• The clustering objective incorporates both data fidelity and structural regularization, promoting segments with consistent MRFs.

Experimental results

Research questions

• RQ1 Can a model-based clustering approach that learns dependency networks (MRFs) from multivariate time series improve interpretability compared to distance-based methods?
• RQ2 How can simultaneous segmentation and clustering be achieved while preserving temporal consistency and capturing structural patterns?
• RQ3 To what extent can TICC discover meaningful, real-world behavioral states (e.g., driving maneuvers) in high-dimensional sensor data?
• RQ4 Does clustering based on structural similarity (partial correlations) outperform clustering based on raw signal alignment, especially in symmetric scenarios like left and right turns?
• RQ5 Can the resulting MRFs be used to interpret cluster semantics through sensor importance metrics such as betweenness centrality?

Key findings

• TICC successfully identified five distinct driving states in an automotive sensor dataset: slowing down, turning, speeding up, driving straight, and navigating curvy roads.
• The method achieved high interpretability, with sensor betweenness centrality scores clearly distinguishing cluster semantics—e.g., Y-Acceleration was most important in turning clusters (#2 and #5).
• Cluster #1, with high Brake Pedal and Velocity centrality, was consistently assigned to slowing down, while cluster #4, dominated by Velocity and Gas Pedal, corresponded to cruising on straight roads.
• TICC correctly grouped both left and right turns into the same cluster due to structural similarity in sensor dependencies, unlike distance-based methods that would treat them as distinct.
• The algorithm demonstrated robustness to symmetric variations, as shown by consistent cluster assignments across left and right turns in the same driving session.
• On synthetic data, TICC outperformed state-of-the-art baselines in clustering accuracy and structural recovery, validating its effectiveness in recovering true underlying patterns.