[Paper Review] DFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation

Jie Yang, Yong Shi+1 more|arXiv (Cornell University)|Dec 13, 2020

Anomaly Detection Techniques and Applications35 references47 citations

TL;DR

DFR introduces a multi-scale regional feature generator from pre-trained CNNs and a convolutional autoencoder to detect and segment anomalies in images, including very small regions, in an unsupervised manner.

ABSTRACT

Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect visual variations, such as defects on manufacturing products. This paper proposes an effective unsupervised anomaly segmentation approach that can detect and segment out the anomalies in small and confined regions of images. Concretely, we develop a multi-scale regional feature generator that can generate multiple spatial context-aware representations from pre-trained deep convolutional networks for every subregion of an image. The regional representations not only describe the local characteristics of corresponding regions but also encode their multiple spatial context information, making them discriminative and very beneficial for anomaly detection. Leveraging these descriptive regional features, we then design a deep yet efficient convolutional autoencoder and detect anomalous regions within images via fast feature reconstruction. Our method is simple yet effective and efficient. It advances the state-of-the-art performances on several benchmark datasets and shows great potential for real applications.

Motivation & Objective

• Motivate robust unsupervised anomaly segmentation for defects that appear in small, confined image regions.
• Propose a multi-scale regional feature generator that yields dense, context-aware regional representations from pre-trained CNNs.
• Develop a lightweight convolutional autoencoder to reconstruct regional features and identify anomalies via reconstruction error.
• Enable efficient inference by leveraging pre-trained features and a single forward pass through the pipeline.
• Demonstrate state-of-the-art performance on benchmark datasets while maintaining computational efficiency.

Proposed method

• Extract hierarchical, multi-scale features from a pre-trained CNN to capture local and global context.
• Align and aggregate feature maps across layers to form a dense multi-scale regional representation for each image region.
• Train a convolutional autoencoder on these regional features to learn normal-region reconstructions only.
• Compute region-wise reconstruction errors as anomaly scores and upsample to pixel-wise maps for segmentation.
• Determine an adaptive threshold based on an acceptable false positive rate to binarize anomaly maps.

Experimental results

Research questions

• RQ1 Can multi-scale, region-focused CNN features improve unsupervised anomaly segmentation, especially for small defects?
• RQ2 Does reconstructing deep feature representations with a CAE yield accurate pixel-level anomaly maps without anomaly examples during training?
• RQ3 How does the proposed DFR compare to existing reconstruction- and feature-based methods on standard benchmarks?
• RQ4 Is the approach computationally efficient enough for practical deployment in manufacturing settings?

Key findings

• Outperforms several state-of-the-art baselines on the MVTec AD dataset across most object and texture categories.
• Demonstrates that multi-scale regional features from pre-trained CNNs, when fused and reconstructed, yield effective anomaly localization.
• The approach achieves competitive ROC-AUC and PRO-AUC metrics compared to ST models while using a single forward pass and fewer training complexities.
• Multi-scale modeling contributes to performance gains, with 16-scale representation yielding strong results.
• Training only the CAE while freezing the pre-trained CNNs makes the method efficient and practical for real-world use.