[Paper Review] StackVAE-G: An efficient and interpretable model for time series anomaly detection

Wenkai Li, Wenbo Hu+3 more|arXiv (Cornell University)|May 18, 2021

Anomaly Detection Techniques and Applications27 references4 citations

TL;DR

This paper proposes StackVAE-G, a novel variational autoencoder with graph neural networks for efficient and interpretable multivariate time series anomaly detection. By combining weight-sharing block-wise reconstruction with learnable sparse graph structures, the model reduces computational cost while maintaining high detection accuracy and enabling interpretable failure diagnosis through learned channel interrelations.

ABSTRACT

Recent studies have shown that autoencoder-based models can achieve superior performance on anomaly detection tasks due to their excellent ability to fit complex data in an unsupervised manner. In this work, we propose a novel autoencoder-based model, named StackVAE-G that can significantly bring the efficiency and interpretability to multivariate time series anomaly detection. Specifically, we utilize the similarities across the time series channels by the stacking block-wise reconstruction with a weight-sharing scheme to reduce the size of learned models and also relieve the overfitting to unknown noises in the training data. We also leverage a graph learning module to learn a sparse adjacency matrix to explicitly capture the stable interrelation structure among multiple time series channels for the interpretable pattern reconstruction of interrelated channels. Combining these two modules, we introduce the stacking block-wise VAE (variational autoencoder) with GNN (graph neural network) model for multivariate time series anomaly detection. We conduct extensive experiments on three commonly used public datasets, showing that our model achieves comparable (even better) performance with the state-of-the-art modelsand meanwhile requires much less computation and memory cost. Furthermore, we demonstrate that the adjacency matrix learned by our model accurately captures the interrelation among multiple channels, and can provide valuable information for failure diagnosis applications.

Motivation & Objective

• Address the inefficiency and lack of interpretability in existing autoencoder-based anomaly detection models for multivariate time series.
• Reduce model complexity and overfitting by exploiting cross-channel similarities through weight-sharing in block-wise reconstruction.
• Capture stable interrelation structures among time series channels to improve reconstruction accuracy and enable fault diagnosis.
• Develop a memory- and computation-efficient model suitable for deployment in resource-constrained environments like edge devices.
• Enable interpretable anomaly diagnosis by learning a meaningful, sparse adjacency matrix reflecting true channel relationships.

Proposed method

• Implement a stacking block-wise reconstruction framework using variational autoencoders (VAEs) to reconstruct multiple time steps in a single forward pass.
• Apply weight-sharing across time blocks to reduce model parameters and mitigate overfitting to noise in training data.
• Integrate a graph neural network (GNN) module to learn a sparse adjacency matrix representing stable interrelations among time series channels.
• Introduce a graph learning loss function (L_Graph) to guide the GNN in learning a meaningful, interpretable graph structure from data.
• Train the end-to-end model using reconstruction loss and the graph learning loss to jointly optimize reconstruction quality and structural consistency.
• Use the learned adjacency matrix to identify neighboring channels for each time series, enabling interpretable anomaly diagnosis.

Experimental results

Research questions

• RQ1 Can block-wise reconstruction with weight-sharing reduce model complexity and improve efficiency without sacrificing anomaly detection performance?
• RQ2 Can a learnable sparse graph structure effectively capture stable interrelations among multivariate time series channels for improved reconstruction and interpretability?
• RQ3 Does the integration of graph learning loss lead to a more interpretable and accurate adjacency matrix compared to unsupervised or pre-trained graph methods?
• RQ4 How does StackVAE-G compare to state-of-the-art models in terms of detection performance, inference cost, and memory usage on standard benchmarks?
• RQ5 Can the learned graph structure support meaningful failure diagnosis by distinguishing single-channel sensor errors from multi-channel systemic failures?

Key findings

• StackVAE-G achieves state-of-the-art or comparable anomaly detection performance on three benchmark datasets: SMD, SMAP, and MSL, with higher F1-scores on several subsets.
• The model significantly reduces computational and memory costs compared to existing autoencoder-based models, making it suitable for edge deployment.
• The graph learning loss is essential for learning a meaningful and interpretable adjacency matrix; without it, the model fails to capture true channel relationships.
• The learned adjacency matrix accurately reflects the true interrelation structure among channels, as validated by visual inspection and ablation studies.
• The model enables interpretable failure diagnosis: systemic anomalies are detected when multiple neighboring channels deviate, while sensor errors are identified when only one channel is anomalous.
• Ablation studies confirm that removing the graph learning loss results in an uninterpretable graph structure, demonstrating the necessity of the proposed loss for reliable structure learning.