[Paper Review] DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data

Lingrui Yu|arXiv (Cornell University)|Feb 17, 2023

Anomaly Detection Techniques and Applications59 references6 citations

TL;DR

DTAAD proposes a lightweight, unsupervised anomaly detection model for multivariate time series using a Dual TCN-Attention network with an autoregressive autoencoder and feedback mechanism. It achieves 8.38% higher F1 scores and 99% faster training than state-of-the-art baselines, demonstrating strong performance and efficiency on six benchmark datasets.

ABSTRACT

Anomaly detection techniques enable effective anomaly detection and diagnosis in multi-variate time series data, which are of major significance for today's industrial applications. However, establishing an anomaly detection system that can be rapidly and accurately located is a challenging problem due to the lack of anomaly labels, the high dimensional complexity of the data, memory bottlenecks in actual hardware, and the need for fast reasoning. In this paper, we propose an anomaly detection and diagnosis model, DTAAD, based on Transformer and Dual Temporal Convolutional Network (TCN). Our overall model is an integrated design in which an autoregressive model (AR) combines with an autoencoder (AE) structure. Scaling methods and feedback mechanisms are introduced to improve prediction accuracy and expand correlation differences. Constructed by us, the Dual TCN-Attention Network (DTA) uses only a single layer of Transformer encoder in our baseline experiment, belonging to an ultra-lightweight model. Our extensive experiments on seven public datasets validate that DTAAD exceeds the majority of currently advanced baseline methods in both detection and diagnostic performance. Specifically, DTAAD improved F1 scores by $8.38\%$ and reduced training time by $99\%$ compared to the baseline. The code and training scripts are publicly available on GitHub at https://github.com/Yu-Lingrui/DTAAD.

Motivation & Objective

• To address the challenge of unsupervised anomaly detection in high-dimensional, multivariate time series with limited labeled data.
• To design a model that captures complex temporal dependencies and cross-variable correlations efficiently.
• To reduce training time and memory usage while maintaining high detection accuracy for real-time industrial deployment.
• To improve anomaly localization and diagnosis by integrating feedback mechanisms and attention mechanisms.

Proposed method

• The model combines an autoregressive autoencoder (AR-AE) with a dual-branch Temporal Convolutional Network (TCN) architecture: local TCN and global TCN for capturing short- and long-range dependencies.
• A single-layer Transformer encoder is used to model long-range temporal patterns, with multi-head self-attention to enhance sequence representation.
• A feedback mechanism is introduced to refine predictions by iteratively correcting reconstruction errors and improving feature learning.
• Loss weighting is applied across two decoders (local and global) to balance learning and expand correlation differences.
• Causal and dilated convolutions are used in TCN blocks to maintain causality and capture long-range context efficiently.
• The model is trained in an unsupervised manner using only future prediction loss, with early stopping and AdamW optimization.

Experimental results

Research questions

• RQ1 Can a lightweight, single-encoder Transformer-based model achieve state-of-the-art performance in multivariate time series anomaly detection?
• RQ2 How effective is the dual TCN-attention design in capturing both local anomalies and global trends?
• RQ3 To what extent does the feedback mechanism improve detection accuracy and generalization across diverse datasets?
• RQ4 Can the model reduce training time by 99% compared to SOTA while maintaining or improving F1 scores?

Key findings

• DTAAD improved F1 scores by 8.38% compared to the strongest baseline on complete datasets, with a 99% reduction in training time.
• The model achieved an average F1 score of 0.8390 on the SMD dataset and 0.8941 on WADI, outperforming all ablated variants.
• The TCN_Global and TCN_Local components contributed most significantly to performance, with their removal causing the largest drop in F1 and AUC scores.
• The window size of 10 provided the optimal balance between contextual information capture and training efficiency, with performance peaking at this setting.
• Attention visualization confirmed that global attention focuses on overall trends while local attention highlights short-term anomalies, validating the model's diagnostic capability.
• The model generalized well across six public datasets, including WADI, SMD, and MSDS, demonstrating robustness and scalability.