[論文レビュー] GDformer: Going Beyond Subsequence Isolation for Multivariate Time Series Anomaly Detection

Qingxiang Liu, Chenghao Liu+3 more|ArXiv.org|Jan 30, 2025

Anomaly Detection Techniques and Applications被引用数 3

ひとこと要約

GDformer は、系列全体の正常表現を学習するためのプロトタイプを持つグローバル辞書強化クロスアテンション Transformer を導入し、多変量時系列の教師なし異常検知で最先端の結果を5つのベンチマークで達成します。

ABSTRACT

Unsupervised anomaly detection of multivariate time series is a challenging task, given the requirements of deriving a compact detection criterion without accessing the anomaly points. The existing methods are mainly based on reconstruction error or association divergence, which are both confined to isolated subsequences with limited horizons, hardly promising unified series-level criterion. In this paper, we propose the Global Dictionary-enhanced Transformer (GDformer) with a renovated dictionary-based cross attention mechanism to cultivate the global representations shared by all normal points in the entire series. Accordingly, the cross-attention maps reflect the correlation weights between the point and global representations, which naturally leads to the representation-wise similarity-based detection criterion. To foster more compact detection boundary, prototypes are introduced to capture the distribution of normal point-global correlation weights. GDformer consistently achieves state-of-the-art unsupervised anomaly detection performance on five real-world benchmark datasets. Further experiments validate the global dictionary has great transferability among various datasets. The code is available at https://github.com/yuppielqx/GDformer.

研究の動機と目的

• Motivate unsupervised anomaly detection for multivariate time series beyond subsequence isolation.
• Learn global representations shared by normal points across the entire series.
• Provide a compact, similarity-based anomaly detection criterion using prototypes.
• Demonstrate transferability of the learned global dictionary across datasets.

提案手法

• Replace standard self-attention with a dictionary-based cross-attention mechanism that uses a global Key/Value dictionary of size N per layer.
• Introduce a similarity-evaluation branch with P prototypes to capture normal point–global correlation patterns.
• Combine reconstruction loss with a similarity-discrepancy loss to train both the dictionary and prototypes (L_total = L_c - lambda L_s).
• Compute AnomalyScore from cross-attention similarity to prototypes and apply a series-level threshold for detection.
• Input subsequences are masked with a fixed probability, normalized, and embedded before transformer layers.

Figure 1: How to derive the detection criterion. Left : AnomalyTrans and DCdctector learn intra-subsequence point-wise association and derive the detection criterion by combining subsequence-level anomaly scores. Right : Our proposal cultivates global normal representations manifested as dictionary

実験結果

リサーチクエスチョン

• RQ1 Can a global dictionary and cross-attention learn series-level normal representations that improve anomaly detection over horizon-limited subsequences?
• RQ2 Do prototypes capturing the distribution of normal cross-attention weights enable a compact, reliable series-level detection criterion?
• RQ3 Are dictionary-based approaches transferable across datasets, indicating shared normal temporal patterns?

主な発見

• GDformer achieves state-of-the-art unsupervised anomaly detection on five real-world benchmarks.
• The global dictionary enables low-complexity cross-attention (O(TN)) compared to O(T^2) for standard self-attention, improving efficiency.
• Prototypes plus a similarity-discrepancy loss significantly improve F1 scores compared to reconstruction-only or single-criterion baselines.
• Ablation shows the combination of cross-attention with multi-layer similarity loss yields the best performance.
• Transfer experiments show the global dictionary and prototypes transfer well across datasets, indicating shared normal patterns.

Figure 2: Anomaly scores v.s. detection criterion for different subsequences in AnomalyTrans.