[Paper Review] Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection

David Zimmerer, Simon Köhl+3 more|arXiv (Cornell University)|Dec 14, 2018

Anomaly Detection Techniques and ApplicationsComputer Science25 references82 citations

TL;DR

The paper proposes ceVAE, a combination of Context Encoders and Variational Autoencoders for unsupervised anomaly detection in medical images, yielding improved sample- and pixel-wise anomaly localization and achieving state-of-the-art unsupervised ROC-AUC on BraTS-2017 and ISLES-2015 datasets.

ABSTRACT

Unsupervised learning can leverage large-scale data sources without the need for annotations. In this context, deep learning-based auto encoders have shown great potential in detecting anomalies in medical images. However, state-of-the-art anomaly scores are still based on the reconstruction error, which lacks in two essential parts: it ignores the model-internal representation employed for reconstruction, and it lacks formal assertions and comparability between samples. We address these shortcomings by proposing the Context-encoding Variational Autoencoder (ceVAE) which combines reconstruction- with density-based anomaly scoring. This improves the sample- as well as pixel-wise results. In our experiments on the BraTS-2017 and ISLES-2015 segmentation benchmarks, the ceVAE achieves unsupervised ROC-AUCs of 0.95 and 0.89, respectively, thus outperforming state-of-the-art methods by a considerable margin.

Motivation & Objective

• Motivate unsupervised anomaly detection in medical imaging due to lack of annotations and need for sample- and pixel-level localization.
• Improve anomaly scoring by incorporating model-internal latent deviations alongside reconstruction errors.
• Provide a pixel-wise anomaly localization mechanism with formal, comparable scores.
• Demonstrate state-of-the-art unsupervised segmentation performance on public benchmarks BraTS-2017 and ISLES-2015.

Proposed method

• Introduce ceVAE by merging Context Encoder (CE) with a Variational Autoencoder (VAE) using shared encoder weights.
• CE branch: apply context-encoding noise by masking input regions and reconstruct the perturbed input with a CE objective to yield robust, semantically meaningful representations.
• VAE branch: use a standard VAE with a latent prior p(z) and KL-divergence L_KL plus reconstruction loss L_rec_VAE to obtain per-sample likelihood estimates.
• Combine CE and VAE into a joint objective L_ceVAE = L_KL + L_rec_VAE + L_rec_CE to capture both latent deviations and calibrated reconstruction errors.
• Anomaly scores: compute sample-wise anomaly score as log p(x) ≈ L_KL + L_rec_VAE; compute pixel-wise scores by fusing reconstruction error and the gradient-derived derivative of the ELBO back-propagated to the input (via backpropagation of the KL term).
• Key equations referenced: (2) ELBO, (4) standard VAE loss, (5) L_VAE, (6) L_ceVAE, (7) sample-wise log probability, (8) pixel-wise anomaly score.

Experimental results

Research questions

• RQ1 Can ceVAE improve unsupervised anomaly detection over reconstruction-based AEs and vanilla VAEs in medical imaging?
• RQ2 Does combining CE with VAE provide better pixel-wise localization and sample-wise scoring than either component alone?
• RQ3 How does integrating the KL-divergence from the posterior to the prior help in outlining abnormal regions at the pixel level?
• RQ4 Are the proposed scores calibrated and comparable across samples for unsupervised anomaly detection on public benchmarks?

Key findings

• ceVAE outperforms OC-SVM and various AE-based baselines on BraTS-2017 and ISLES-2015 in pixel-wise anomaly detection and sample-wise ROC-AUC.
• Pixel-wise fusion of reconstruction error and KL-divergence gradient yields superior segmentation performance across datasets.
• Combining CE with VAE provides regularization benefits and avoids posterior collapse, leading to more discriminative latent representations.
• The method achieves unsupervised ROC-AUCs of 0.95 (BraTS-2017) and 0.89 (ISLES-2015).
• Using CE alone excels on reconstruction-based metrics, while VAE alone excels on gradient-based metrics; their combination (ceVAE) consistently yields the best results.