[Paper Review] MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks

Dan Li, Dacheng Chen+4 more|arXiv (Cornell University)|Jan 15, 2019

Anomaly Detection Techniques and ApplicationsComputer Science25 references138 citations

TL;DR

MAD-GAN uses an LSTM-based GAN to model multivariate time series and detects anomalies via a combined discrimination and reconstruction score, evaluated on SWaT and WADI cyber-attack datasets.

ABSTRACT

The prevalence of networked sensors and actuators in many real-world systems such as smart buildings, factories, power plants, and data centers generate substantial amounts of multivariate time series data for these systems. The rich sensor data can be continuously monitored for intrusion events through anomaly detection. However, conventional threshold-based anomaly detection methods are inadequate due to the dynamic complexities of these systems, while supervised machine learning methods are unable to exploit the large amounts of data due to the lack of labeled data. On the other hand, current unsupervised machine learning approaches have not fully exploited the spatial-temporal correlation and other dependencies amongst the multiple variables (sensors/actuators) in the system for detecting anomalies. In this work, we propose an unsupervised multivariate anomaly detection method based on Generative Adversarial Networks (GANs). Instead of treating each data stream independently, our proposed MAD-GAN framework considers the entire variable set concurrently to capture the latent interactions amongst the variables. We also fully exploit both the generator and discriminator produced by the GAN, using a novel anomaly score called DR-score to detect anomalies by discrimination and reconstruction. We have tested our proposed MAD-GAN using two recent datasets collected from real-world CPS: the Secure Water Treatment (SWaT) and the Water Distribution (WADI) datasets. Our experimental results showed that the proposed MAD-GAN is effective in reporting anomalies caused by various cyber-intrusions compared in these complex real-world systems.

Motivation & Objective

• Motivate anomaly detection in cyber-physical systems using multivariate time-series data with limited labeled anomalies.
• Propose MAD-GAN to capture temporal and cross-variable dependencies via LSTM-based GANs.
• Develop a DR-score that combines discrimination and reconstruction losses for anomaly detection.
• Evaluate MAD-GAN on real-world CPS datasets SWaT and WADI to assess intrusion-detection performance.

Proposed method

• Construct GAN with LSTM-RNN generator and discriminator to model normal multivariate time series as sequences.
• Split multivariate time series into overlapping sub-sequences with a sliding window to capture temporal dynamics.
• Train G and D in a standard GAN minimax framework to learn the distribution of normal data.
• Use the trained generator for reconstruction-based anomaly scoring and the discriminator for discrimination-based scoring.
• Combine reconstruction and discrimination losses into the DR-Score to detect anomalies across sub-sequences and map them back to the original time series.
• Evaluate using precision, recall, and F1 on anomaly detection across multiple window sizes; compare MAD-GAN against PCA, KNN, Feature Bagging, AE, and EGAN.

Experimental results

Research questions

• RQ1 Can MAD-GAN effectively model multivariate time-series dependencies for unsupervised anomaly detection in CPS data?
• RQ2 Does leveraging both the GAN discriminator and generator (via DR-Score) improve anomaly detection over single-aspect methods?
• RQ3 How does MAD-GAN perform on real-world CPS datasets SWaT and WADI under cyber-attacks compared to other unsupervised methods?

Key findings

• MAD-GAN achieves near 100% precision and high recall on SWaT for the best F1-balanced configuration, outperforming several baselines.
• On WADI, MAD-GAN yields high recall (up to 99.99% in some configurations) with varying precision, indicating effective intrusion detection with tolerable false alarms.
• Across datasets, MAD-GAN generally outperforms PCA, KNN, FB, AE, and EGAN in at least one evaluation metric, with the best F1 on SWaT reaching 0.70 and on KDDCUP99 reaching 0.90.
• The study demonstrates multivariate modeling benefits; multivariate GAN training (as opposed to univariate) yields faster and more realistic sample generation (MMD convergence).
• DR-Score effectively fuses reconstruction residuals and discrimination outputs to detect anomalies in sub-sequences and across time steps.