[Paper Review] DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection

Yiyuan Yang, Chaoli Zhang+3 more|arXiv (Cornell University)|Jun 17, 2023

Anomaly Detection Techniques and ApplicationsComputer Science76 references10 citations

TL;DR

DCdetector introduces a dual-branch, dual-attention contrastive framework for unsupervised time series anomaly detection that learns permutation-invariant representations to distinguish normal points from anomalies without reconstruction losses.

ABSTRACT

Time series anomaly detection is critical for a wide range of applications. It aims to identify deviant samples from the normal sample distribution in time series. The most fundamental challenge for this task is to learn a representation map that enables effective discrimination of anomalies. Reconstruction-based methods still dominate, but the representation learning with anomalies might hurt the performance with its large abnormal loss. On the other hand, contrastive learning aims to find a representation that can clearly distinguish any instance from the others, which can bring a more natural and promising representation for time series anomaly detection. In this paper, we propose DCdetector, a multi-scale dual attention contrastive representation learning model. DCdetector utilizes a novel dual attention asymmetric design to create the permutated environment and pure contrastive loss to guide the learning process, thus learning a permutation invariant representation with superior discrimination abilities. Extensive experiments show that DCdetector achieves state-of-the-art results on multiple time series anomaly detection benchmark datasets. Code is publicly available at https://github.com/DAMO-DI-ML/KDD2023-DCdetector.

Motivation & Objective

• Motivate unsupervised anomaly detection in multivariate time series by leveraging contrastive representation learning.
• Propose a lightweight dual-branch architecture with dual attention to distinguish normal patterns from anomalies.
• Eliminate reliance on reconstruction loss and negative samples to reduce distraction from anomalies.
• Demonstrate state-of-the-art performance on diverse real-world datasets and provide analysis on avoiding collapse.

Proposed method

• Use a patching-based, channel-independent attention module to capture local and cross-channel temporal dependencies.
• Implement a dual-attention contrastive structure that processes two views of the data: patch-wise and in-patch representations with shared weights.
• Optimize a symmetric KL-divergence–based loss that aligns the two views while preventing trivial collapse without negative samples.
• Compute an anomaly score as the sum of KL divergences between the two representations with stop-gradient operations to enforce asymmetry.
• Adopt a multi-scale patching design to reduce information loss and improve representation richness.

Experimental results

Research questions

• RQ1 Can a dual-branch, dual-attention contrastive framework learn discriminative representations for time series anomalies without reconstruction loss?
• RQ2 Do permutation-invariant representations across patch-wise and in-patch views effectively separate normal points from anomalies in multivariate time series?
• RQ3 How does multi-scale patching affect information preservation and anomaly discrimination in DCdetector?
• RQ4 Is the model robust to collapse in a purely self-supervised setting without negative samples?

Key findings

• DCdetector achieves competitive or superior performance to state-of-the-art methods on seven multivariate and one univariate time series anomaly detection benchmarks.
• The method uses a two-branch, shared-weight architecture that learns permutation-invariant representations for normal points while anomalies exhibit larger representation discrepancies across views.
• A reconstruction-free, contrastive objective with stop-gradient reduces distractions from anomalies and demonstrates robustness against trivial collapse.
• Multi-scale patching improves information retention and aids in capturing local semantic patterns, contributing to improved anomaly detection performance.
• Empirical results show strong performance across diverse datasets, including real-world multivariate and univariate time series.