[Paper Review] Sub-Image Anomaly Detection with Deep Pyramid Correspondences

Niv Cohen, Yedid Hoshen|arXiv (Cornell University)|May 5, 2020

Anomaly Detection Techniques and Applications45 references325 citations

TL;DR

SPADE uses a multi-resolution feature pyramid and nearest-neighbor image correspondences to detect and localize anomalies within images without extensive training.

ABSTRACT

Nearest neighbor (kNN) methods utilizing deep pre-trained features exhibit very strong anomaly detection performance when applied to entire images. A limitation of kNN methods is the lack of segmentation map describing where the anomaly lies inside the image. In this work we present a novel anomaly segmentation approach based on alignment between an anomalous image and a constant number of the similar normal images. Our method, Semantic Pyramid Anomaly Detection (SPADE) uses correspondences based on a multi-resolution feature pyramid. SPADE is shown to achieve state-of-the-art performance on unsupervised anomaly detection and localization while requiring virtually no training time.

Motivation & Objective

• Address the challenge of locating and segmenting anomalies within images when only normal data is available during training.
• Develop a fast, training-light anomaly detection framework that provides pixel-level localization.
• Leverage pre-trained deep features and a multi-resolution feature pyramid to enable robust correspondences across images.

Proposed method

• Extract pre-trained deep features (ImageNet-resnet) for whole images and per-pixel locations.
• Retrieve K nearest normal images using image-level features to identify normal context.
• Construct a pixel-level feature gallery from the K nearest normals and compute dense correspondences for each pixel.
• Compute an anomaly score per pixel as the average distance to its kappa nearest features in the gallery.
• Use a feature pyramid by concatenating multi-level ResNet features to achieve robust pixel correspondences across contexts.
• Label pixels as anomalous if their local feature distance exceeds a threshold, followed by Gaussian smoothing for the final map.

Experimental results

Research questions

• RQ1 Can a KNN-based, correspondence-driven approach localize sub-image anomalies without explicit anomaly training data?
• RQ2 Does incorporating a multi-scale feature pyramid improve pixel-level localization accuracy compared to single-scale features?
• RQ3 How does the method perform on industrial (MVTech) and surveillance (Shanghai Tech Campus) datasets in image- and pixel-level metrics?
• RQ4 What is the impact of using pre-trained ImageNet features versus self-supervised learned features for anomaly detection and localization?

Key findings

• SPADE achieves state-of-the-art performance on sub-image anomaly detection and localization on MVTech and Shanghai Tech Campus datasets.
• Using a multi-level feature pyramid improves pixel-level localization accuracy over single-layer features.
• The approach yields strong image-level anomaly detection and significantly better pixel-level ROCAUC and PRO scores than several autoencoder-based methods.
• The method requires virtually no training time beyond extracting pre-trained features and nearest-neighbor search.