[Paper Review] Deep Nearest Neighbor Anomaly Detection

Liron Bergman, Niv Cohen+1 more|arXiv (Cornell University)|Feb 24, 2020

Anomaly Detection Techniques and Applications26 references108 citations

TL;DR

The paper shows that a simple kNN anomaly detector using Imagenet-pretrained ResNet features (DN2) outperforms state-of-the-art self-supervised and deep methods across unimodal, multimodal, unsupervised, and group anomaly tasks, with low sample complexity and no training stage.

ABSTRACT

Nearest neighbors is a successful and long-standing technique for anomaly detection. Significant progress has been recently achieved by self-supervised deep methods (e.g. RotNet). Self-supervised features however typically under-perform Imagenet pre-trained features. In this work, we investigate whether the recent progress can indeed outperform nearest-neighbor methods operating on an Imagenet pretrained feature space. The simple nearest-neighbor based-approach is experimentally shown to outperform self-supervised methods in: accuracy, few shot generalization, training time and noise robustness while making fewer assumptions on image distributions.

Motivation & Objective

• Motivate a simple, strong baseline for image anomaly detection using pretrained features and kNN.
• Demonstrate DN2's performance across semi-supervised, unsupervised, and group anomaly settings.
• Show robustness to training impurities, small datasets, and data distributions unlike Imagenet.
• Compare DN2 to self-supervised and deep learning based anomaly detectors.
• Provide practical guidance on when and how to apply DN2 (and its limitations).

Proposed method

• Embed all images with a pretrained feature extractor (Imagenet-pretrained ResNet).
• Compute k nearest neighbor distances in feature space as anomaly scores.
• For semi-supervised: classify by thresholding kNN distance to training embeddings.
• For unsupervised: perform a cleaning stage by removing high-density-outlier images before re-running DN2.
• For group anomaly detection: pool image features within a set by averaging and apply DN2 on the group embedding.
• Discuss practical speedups via clustering training features for approximate kNN retrieval.

Experimental results

Research questions

• RQ1 Does kNN on pretrained image features outperform self-supervised and deep feature methods for anomaly detection?
• RQ2 How does DN2 perform under semi-supervised, unsupervised, and group anomaly settings?
• RQ3 What is the impact of network depth, number of neighbors, and data invariances on DN2 performance?
• RQ4 Is DN2 effective with small training sets and across diverse image domains?
• RQ5 Can DN2 robustly handle noisy or contaminated training data?

Key findings

• DN2 often outperforms state-of-the-art anomaly detection methods across several datasets (e.g., CIFAR10, Fashion-MNIST, CIFAR100).
• Using Imagenet-pretrained ResNet features yields strong locality in feature space, aiding kNN-based detection.
• Deeper networks improve DN2 performance; two neighbors often suffice for optimal ROCAUC.
• DN2 remains effective with very small training sets, unlike some self-supervised baselines.
• A cleaning stage before unsupervised DN2 significantly mitigates performance loss due to impurities in the training data.
• DN2 excels in group anomaly detection by mean-pooling group features before applying DN2; outperforms simple concatenation baselines.