[Paper Review] Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series

Dan Li, Dacheng Chen+2 more|arXiv (Cornell University)|Sep 13, 2018

Anomaly Detection Techniques and Applications41 references219 citations

TL;DR

This paper proposes GAN-AD, an unsupervised anomaly detection method for multivariate time series from cyber-physical systems, using LSTM-based generator and discriminator to model normal behavior and detect anomalies via residual and discrimination losses.

ABSTRACT

Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with networked sensors and actuators that are targets for cyber-attacks. Conventional detection techniques are unable to deal with the increasingly dynamic and complex nature of the CPSs. On the other hand, the networked sensors and actuators generate large amounts of data streams that can be continuously monitored for intrusion events. Unsupervised machine learning techniques can be used to model the system behaviour and classify deviant behaviours as possible attacks. In this work, we proposed a novel Generative Adversarial Networks-based Anomaly Detection (GAN-AD) method for such complex networked CPSs. We used LSTM-RNN in our GAN to capture the distribution of the multivariate time series of the sensors and actuators under normal working conditions of a CPS. Instead of treating each sensor's and actuator's time series independently, we model the time series of multiple sensors and actuators in the CPS concurrently to take into account of potential latent interactions between them. To exploit both the generator and the discriminator of our GAN, we deployed the GAN-trained discriminator together with the residuals between generator-reconstructed data and the actual samples to detect possible anomalies in the complex CPS. We used our GAN-AD to distinguish abnormal attacked situations from normal working conditions for a complex six-stage Secure Water Treatment (SWaT) system. Experimental results showed that the proposed strategy is effective in identifying anomalies caused by various attacks with high detection rate and low false positive rate as compared to existing methods.

Motivation & Objective

• Motivate robust anomaly detection in complex CPSs with high-dimensional, dynamically interacting sensor/actuator streams.
• Develop a GAN-based framework that jointly models normal multivariate time series using LSTM-RNNs.
• Leverage both generator-based residuals and discriminator-based discrimination for anomaly scoring.
• Evaluate GAN-AD on the SWaT water treatment testbed under various cyber-attacks to show improved detection and reduced false positives.

Proposed method

• Use a GAN where both generator and discriminator are LSTM-RNNs to capture non-linear temporal correlations in multivariate time series.
• Train G to generate realistic normal sequences and D to distinguish real from generated sequences.
• Compute an anomaly score combining residual loss (distance between real testing data and generator reconstruction in latent space) and discrimination loss (D’s output on testing data).
• Map high-dimensional testing data to a latent space to obtain corresponding G(Z) and residuals.
• Optionally apply PCA to reduce dimensionality before feeding data to GAN-AD to manage computational load.
• Followed by anomaly scoring S_t = lambda * Residual(X_t) + (1 - lambda) * D(X_t).
• Use Adam/gradient-based optimization for network training and add a latent-space mapping step to identify Z^k that best reconstructs testing data.

Experimental results

Research questions

• RQ1 Can a GAN with LSTM-RNN components model the distribution of normal multivariate CPS time series to enable unsupervised anomaly detection?
• RQ2 Does leveraging both the GAN generator (via reconstruction residuals) and the GAN discriminator improve anomaly detection performance over traditional methods in CPS data?
• RQ3 How does the proposed GAN-AD perform on a real, multi-stage CPS testbed under cyber-attacks compared to PCA-based or SPC methods?
• RQ4 What is the impact of multivariate modeling (as opposed to univariate) on anomaly detection effectiveness in CPS data?

Key findings

• GAN-AD achieves competitive to superior anomaly detection performance on the SWaT dataset across several targets when compared to CUSUM and SPE-based PCA methods.
• Univariate detection is generally weaker with higher false positives and lower precision/recall than multivariate GAN-AD.
• For selected points, GAN-AD substantially improves precision/recall trade-offs (e.g., LIT-101: GAN-AD Precision 50.00% and Recall 1.75%, vs. CUSUM; overall All: GAN-AD 1 achieves Accu 90.57%, Pre 85.71%, Rec 7.20%, F1 0.13, FPR 0.13).
• Multivariate GAN-AD (when using multiple streams) converges to realistic data distributions faster and shows lower MMD during training, indicating effective learning of joint dynamics.
• The first five principal components in PCA-based projection still allow GAN-AD to operate effectively, with high anomaly scores observed when attacks occur across multiple sensors/actuators.