[論文レビュー] Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection

Dong Gong, Lingqiao Liu+5 more|arXiv (Cornell University)|Apr 4, 2019

Anomaly Detection Techniques and Applications参考文献 49被引用数 143

ひとこと要約

MemAE enhances autoencoders with a memory module and sparse attention to memorize normal patterns, improving unsupervised anomaly detection across image, video, and cybersecurity data.

ABSTRACT

Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal ones, which is adopted as a criterion for identifying anomalies. However, this assumption does not always hold in practice. It has been observed that sometimes the autoencoder "generalizes" so well that it can also reconstruct anomalies well, leading to the miss detection of anomalies. To mitigate this drawback for autoencoder based anomaly detector, we propose to augment the autoencoder with a memory module and develop an improved autoencoder called memory-augmented autoencoder, i.e. MemAE. Given an input, MemAE firstly obtains the encoding from the encoder and then uses it as a query to retrieve the most relevant memory items for reconstruction. At the training stage, the memory contents are updated and are encouraged to represent the prototypical elements of the normal data. At the test stage, the learned memory will be fixed, and the reconstruction is obtained from a few selected memory records of the normal data. The reconstruction will thus tend to be close to a normal sample. Thus the reconstructed errors on anomalies will be strengthened for anomaly detection. MemAE is free of assumptions on the data type and thus general to be applied to different tasks. Experiments on various datasets prove the excellent generalization and high effectiveness of the proposed MemAE.

研究の動機と目的

• Motivate and address failure cases of standard deep autoencers where anomalies are reconstructed well despite training on normal data.
• Propose MemAE, a memory-augmented autoencoder that retrieves prototypical normal patterns to constrain reconstruction.
• Demonstrate MemAE’s generality and effectiveness across image, video, and cybersecurity anomaly detection tasks.

提案手法

• Encode input to latent z with an encoder.
• Use z as a query to retrieve relevant memory items via attention-based addressing.
• Aggregate retrieved memory items to form a reconstructed latent representation for the decoder.
• Train end-to-end with a reconstruction loss plus a sparsity-encouraging entropy loss on memory addressing weights.
• Apply a hard shrinkage to encourage sparse memory addressing and re-normalize the weights for ẑ.
• Memory contents are updated during training to capture prototypical normal patterns; at test time, reconstruction uses a small set of normal memory items.

実験結果

リサーチクエスチョン

• RQ1 Can a memory-augmented autoencoder improve unsupervised anomaly detection by constraining reconstructions to prototypical normal patterns?
• RQ2 Does sparse, attention-based memory addressing help prevent overgeneralization and improve anomaly scoring across diverse data domains?
• RQ3 Is MemAE generalizable across images, video, and cybersecurity datasets without data-type specific assumptions?

主な発見

Dataset	OC-SVM	KDE	VAE	PixCNN	DSEBM	AE	MemAE-nonSpar	MemAE
MNIST	0.9499	0.8116	0.9643	0.6141	0.9554	0.9619	0.9725	0.9751
CIFAR-10	0.5619	0.5756	0.5725	0.5450	0.5725	0.5706	0.6058	0.6088

• MemAE generally outperforms several baselines and AE variants on image anomaly detection (MNIST and CIFAR-10) with higher AUC scores.
• Memory-augmented variants (MemAE and MemAE-nonSpar) outperform AE, with sparse addressing providing further gains.
• On video anomaly detection, MemAE achieves higher AUC than competing reconstruction-based methods across UCSD-Ped2, CUHK Avenue, and ShanghaiTech datasets, while maintaining fast per-frame latency (~38 fps).
• MemAE shows strong performance on cybersecurity data (KDDCUP), achieving higher precision, recall, and F1 than several baselines.
• Ablation studies confirm the importance of both the hard shrinkage and entropy-based sparsity loss for best performance.