[Paper Review] Hierarchical Clustering using Auto-encoded Compact Representation for Time-series Analysis

Soma Bandyopadhyay, Anish Datta+1 more|arXiv (Cornell University)|Jan 11, 2021

Time Series Analysis and Forecasting13 references4 citations

TL;DR

This paper proposes HC-AECS, a hierarchical clustering method for time-series that uses a sequence-to-sequence autoencoder to learn a compact latent representation (AECS), significantly reducing computational cost while improving clustering performance. The method applies agglomerative hierarchical clustering with optimized distance measures (Chebyshev, Manhattan, Mahalanobis) and selects the best clustering using the Modified Hubert statistic, outperforming state-of-the-art methods on both univariate and multivariate time-series datasets with up to 47× faster computation than DTW-based alternatives.

ABSTRACT

Getting a robust time-series clustering with best choice of distance measure and appropriate representation is always a challenge. We propose a novel mechanism to identify the clusters combining learned compact representation of time-series, Auto Encoded Compact Sequence (AECS) and hierarchical clustering approach. Proposed algorithm aims to address the large computing time issue of hierarchical clustering as learned latent representation AECS has a length much less than the original length of time-series and at the same time want to enhance its performance.Our algorithm exploits Recurrent Neural Network (RNN) based under complete Sequence to Sequence(seq2seq) autoencoder and agglomerative hierarchical clustering with a choice of best distance measure to recommend the best clustering. Our scheme selects the best distance measure and corresponding clustering for both univariate and multivariate time-series. We have experimented with real-world time-series from UCR and UCI archive taken from diverse application domains like health, smart-city, manufacturing etc. Experimental results show that proposed method not only produce close to benchmark results but also in some cases outperform the benchmark.

Motivation & Objective

• Address the high computational cost of hierarchical clustering on long time-series by learning a compact latent representation.
• Improve clustering performance by combining autoencoder-based representation learning with hierarchical clustering using optimal distance measures.
• Enable effective clustering on both univariate and multivariate time-series, where existing state-of-the-art methods are limited to univariate cases.
• Develop a novel internal validation mechanism using the Modified Hubert statistic to automatically select the best distance measure and clustering configuration.
• Demonstrate robustness and efficiency across diverse real-world time-series from healthcare, manufacturing, and smart-city domains.

Proposed method

• Employ a deep, undercomplete sequence-to-sequence autoencoder (Seq2Seq AE) to learn a compact latent representation (AECS) of time-series, reducing sequence length significantly.
• Apply agglomerative hierarchical clustering on the learned AECS representation to form clusters with reduced computational complexity.
• Evaluate clustering using three distance measures: Chebyshev (CH), Manhattan (MA), and Mahalanobis (ML), with ML used uniquely for internal validation.
• Use the Modified Hubert statistic (𝒯) as an internal clustering validation measure to select the optimal distance measure and corresponding clustering configuration.
• Train the autoencoder end-to-end using reconstruction loss to preserve essential time-series patterns in the compact latent space.
• Integrate the entire pipeline into a unified framework that selects the best clustering based on 𝒯, ensuring robustness across diverse datasets.

Experimental results

Research questions

• RQ1 Can a learned compact representation from a Seq2Seq autoencoder improve the efficiency and performance of hierarchical clustering on long time-series?
• RQ2 Which distance measure (Chebyshev, Manhattan, Mahalanobis) yields the most coherent and well-separated clusters when applied to autoencoded time-series representations?
• RQ3 How does the proposed HC-AECS method compare in clustering accuracy and computational efficiency to state-of-the-art methods like K-Shape and HC-DDTW on univariate and multivariate time-series?
• RQ4 Can the Modified Hubert statistic effectively guide the selection of the optimal distance measure for hierarchical clustering without ground-truth labels?
• RQ5 To what extent does the method scale to real-world time-series with high sequence length, such as in healthcare and industrial IoT applications?

Key findings

• HC-AECS outperforms the benchmark HC-DDTW method on 7 out of 9 univariate time-series datasets from the UCR archive, with a 27× reduction in computation time compared to HC-DDTW.
• On multivariate time-series, HC-AECS significantly outperforms the state-of-the-art MLSTM-FCN model, achieving 0.929 accuracy on the Wafer dataset versus MLSTM-FCN’s 0.906.
• For long sequences like Coffee (286 timesteps) and Beef (470 timesteps), HC-AECS is 16× and 18× faster than K-Shape, respectively, despite K-Shape being limited to univariate data.
• The method achieves a 47× speedup over HC-DDTW on the Adiac dataset, demonstrating strong scalability with increasing sequence length.
• The use of Mahalanobis distance in the Modified Hubert statistic evaluation is a novel contribution that enhances clustering quality and enables effective internal validation.
• HC-AECS achieves competitive or superior Rand Index (RI) scores compared to K-Shape and HC-DDTW across diverse datasets, including ECG200, FordB, and Wafer, confirming its robustness and generalization capability.