[论文解读] TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data

Shreshth Tuli, Giuliano Casale+1 more|arXiv (Cornell University)|Jan 18, 2022

Anomaly Detection Techniques and Applications被引用 112

一句话总结

TranAD 引入了一种基于 transformer 的多变量时间序列异常检测与诊断模型，利用自我条件化、对抗训练和 MAML，实现快速、鲁棒的检测，数据有限时表现良好。

ABSTRACT

Efficient anomaly detection and diagnosis in multivariate time-series data is of great importance for modern industrial applications. However, building a system that is able to quickly and accurately pinpoint anomalous observations is a challenging problem. This is due to the lack of anomaly labels, high data volatility and the demands of ultra-low inference times in modern applications. Despite the recent developments of deep learning approaches for anomaly detection, only a few of them can address all of these challenges. In this paper, we propose TranAD, a deep transformer network based anomaly detection and diagnosis model which uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD uses focus score-based self-conditioning to enable robust multi-modal feature extraction and adversarial training to gain stability. Additionally, model-agnostic meta learning (MAML) allows us to train the model using limited data. Extensive empirical studies on six publicly available datasets demonstrate that TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training. Specifically, TranAD increases F1 scores by up to 17%, reducing training times by up to 99% compared to the baselines.

研究动机与目标

• 提升高维多变量时间序列的异常检测与诊断。
• 解决工业场景中标注数据匮乏和超低推理时间的需求。
• 利用 Transformer 架构高效捕捉长距离的时序趋势。
• 引入稳定的对抗训练和自我条件化以提升鲁棒性和泛化能力。
• 通过模型无关的元学习（MAML）在有限数据下证明有效性。

提出的方法

• 采用基于 Transformer 的编码器–解码器架构，并结合注意力机制来建模时序趋势。
• 应用两阶段自我条件化以放大重构偏差并聚焦于信息量大的子序列。
• 结合带有两个解码器的对抗训练以稳定学习并提高鲁棒性。
• 使用模型无关的元学习（MAML）在有限数据下进行自适应。
• 将数据预处理为带复制填充的滑动窗口并归一化到 [0,1)。
• 通过两阶段重构和 Peak Over Threshold (POT) 进行动态阈值化来推断异常。

实验结果

研究问题

• RQ1 TranAD 在多变量时间序列的异常检测与诊断方面是否优于现有最先进基线？
• RQ2 两阶段自我条件化对抗训练是否提升检测稳定性和对短期时序趋势的敏感性？
• RQ3 MAML 是否能够在标注数据有限的异常检测任务中实现有效学习？
• RQ4 TranAD 是否能够在高维数据上实现快速训练和推理，同时不牺牲准确性？

主要发现

• TranAD 在六个公开数据集上超越基线，F1 分数最高提升至 17%。
• 与竞争方法相比，训练时间最多可降低 99%。
• 两阶段对抗训练结合自我条件化提高鲁棒性并改善对微小偏差的检测。
• MAML 在有限训练数据下也能表现良好，展示快速适应性。
• 注意力和聚焦分数与数据峰值及高偏差段相关联，从而实现逐维度的异常识别。