[論文レビュー] Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features

Naveen Sai Madiraju, Seid M. Sadat+2 more|arXiv (Cornell University)|Feb 4, 2018

Time Series Analysis and Forecasting参考文献 12被引用数 107

ひとこと要約

DTC は時系列の時間次元縮約とクラスタリングをエンドツーエンドの深層フレームワークで同時学習し、時系列自己符号化器と新規クラスタリング層を用い、様々な時系列類似度指標を活用；ベースラインを上回る。

ABSTRACT

Unsupervised learning of time series data, also known as temporal clustering, is a challenging problem in machine learning. Here we propose a novel algorithm, Deep Temporal Clustering (DTC), to naturally integrate dimensionality reduction and temporal clustering into a single end-to-end learning framework, fully unsupervised. The algorithm utilizes an autoencoder for temporal dimensionality reduction and a novel temporal clustering layer for cluster assignment. Then it jointly optimizes the clustering objective and the dimensionality reduction objec tive. Based on requirement and application, the temporal clustering layer can be customized with any temporal similarity metric. Several similarity metrics and state-of-the-art algorithms are considered and compared. To gain insight into temporal features that the network has learned for its clustering, we apply a visualization method that generates a region of interest heatmap for the time series. The viability of the algorithm is demonstrated using time series data from diverse domains, ranging from earthquakes to spacecraft sensor data. In each case, we show that the proposed algorithm outperforms traditional methods. The superior performance is attributed to the fully integrated temporal dimensionality reduction and clustering criterion.

研究の動機と目的

• Motivate unsupervised learning for complex time-series where labels are unavailable or unreliable.
• Develop an end-to-end architecture that jointly optimizes reconstruction and clustering losses.
• Enable flexible temporal similarity metrics within a learnable clustering layer.
• Provide visualization to localize and explain temporal events detected by clusters.
• Demonstrate superior performance over traditional temporal clustering methods across diverse datasets.

提案手法

• Encode time-series into a latent space using a temporal autoencoder (CNN for short-term features + BI-LSTM for multi-scale temporal dynamics).
• Introduce a temporal clustering layer that assigns latent representations to k clusters via a similarity metric.
• Compute cluster assignments using a KL-divergence loss between a soft assignment q and a target distribution p.
• Train end-to-end by jointly minimizing reconstruction loss (MSE) and clustering loss (KL divergence).
• Initialize cluster centroids via hierarchical clustering on latent features and refine them during training.
• Provide heatmap-based visualization to localize informative time regions contributing to cluster assignments.

実験結果

リサーチクエスチョン

• RQ1 Can end-to-end joint optimization of reconstruction and clustering yields superior unsupervised temporal clustering compared to separate steps?
• RQ2 How do different temporal similarity metrics affect clustering performance in a deep temporal clustering framework?
• RQ3 Is the proposed DTC robust across diverse real-world time-series domains and dataset sizes?
• RQ4 Can the model localize temporal events within unlabeled sequences to aid interpretation?

主な発見

Dataset(N, L, r, P)	ACF	DTC ACF	CID	DTC CID	COR	DTC COR	EUCL	DTC EUCL	kshape
NASA MMS (104,1140,1.21,10)	0.51	0.59	0.85	0.93	0.65	0.67	0.56	0.69	0.61
BeetleFly (40,512,1.00,8)	0.55	0.69	0.80	0.892	0.55	0.584	0.55	0.606	0.65
BirdChicken (40,512,1.00,8)	0.70	0.792	0.60	0.732	0.55	0.712	0.55	0.772	0.52
Computers (500,720,1.00,10)	0.58	0.64	0.51	0.68	0.55	0.555	0.55	0.58	0.58
Earthquakes (461,512,0.25,8)	0.508	0.569	0.508	0.588	0.546	0.549	0.546	0.540	0.59
MoteStrain (1272,84,0.86,4)	0.68	0.89	0.57	0.81	0.60	0.93	0.60	0.93	0.88
Phalanges OutlinesCorrect (2658,80,1.77,4)	0.586	0.522	0.501	0.529	0.501	0.525	0.501	0.556	0.56
ProximalPhalanx OutlineCorrect (891,80,2.12,4)	0.52	0.678	0.5	0.66	0.52	0.62	0.52	0.63	0.65
ShapeletSim (200,500,1.00,10)	0.54	0.74	0.83	0.91	0.52	0.55	0.52	0.53	0.56
SonyAIBO RobotSurfaceII (980,65,1.61,5)	0.56	0.72	0.827	0.85	0.57	0.82	0.57	0.83	0.65
SonyAIBO RobotSurface (621,70,0.78,5)	0.74	0.94	0.51	0.81	0.58	0.78	0.58	0.66	0.74
ItalyPower Demand (1096,24,1,4)	0.59	0.63	0.60	0.66	0.54	0.57	0.54	0.61	0.52
WormsTwoClass (258,900,1.37,10)	0.53	0.62	0.59	0.61	0.55	0.56	0.55	0.51	0.55

• Joint end-to-end training significantly improves clustering performance compared to disjoint training (AUC 0.93 vs 0.88 on MMS dataset).
• DTC outperforms baseline temporal clustering methods (k-Shape and hierarchical clustering) across 13 datasets and multiple similarity metrics.
• DTC shows robust improvements across domains with varying sequence lengths and class imbalances.
• Heatmap visualizations localize event locations in time, providing interpretable explanations for cluster assignments.