[Paper Review] SSD: A Unified Framework for Self-Supervised Outlier Detection

Vikash Sehwag, Mung Chiang+1 more|arXiv (Cornell University)|Mar 22, 2021

Anomaly Detection Techniques and Applications53 references44 citations

TL;DR

SSD uses self-supervised representation learning and Mahalanobis distance to detect out-of-distribution samples using only unlabeled in-distribution data, with extensions to few-shot OOD and label-assisted detection that achieve strong results.

ABSTRACT

We ask the following question: what training information is required to design an effective outlier/out-of-distribution (OOD) detector, i.e., detecting samples that lie far away from the training distribution? Since unlabeled data is easily accessible for many applications, the most compelling approach is to develop detectors based on only unlabeled in-distribution data. However, we observe that most existing detectors based on unlabeled data perform poorly, often equivalent to a random prediction. In contrast, existing state-of-the-art OOD detectors achieve impressive performance but require access to fine-grained data labels for supervised training. We propose SSD, an outlier detector based on only unlabeled in-distribution data. We use self-supervised representation learning followed by a Mahalanobis distance based detection in the feature space. We demonstrate that SSD outperforms most existing detectors based on unlabeled data by a large margin. Additionally, SSD even achieves performance on par, and sometimes even better, with supervised training based detectors. Finally, we expand our detection framework with two key extensions. First, we formulate few-shot OOD detection, in which the detector has access to only one to five samples from each class of the targeted OOD dataset. Second, we extend our framework to incorporate training data labels, if available. We find that our novel detection framework based on SSD displays enhanced performance with these extensions, and achieves state-of-the-art performance. Our code is publicly available at https://github.com/inspire-group/SSD.

Motivation & Objective

• Motivate outlier/OOD detection with only unlabeled in-distribution data.
• Develop a self-supervised, cluster-conditioned Mahalanobis detector that leverages in-distribution features.
• Provide extensions to few-shot OOD detection and to incorporate labels when available.

Proposed method

• Train a feature extractor with contrastive self-supervised learning (NT-Xent) on unlabeled in-distribution data.
• Partition in-distribution features into clusters and model each cluster with Mahalanobis distance in feature space.
• Compute outlier scores as the minimum cluster Mahalanobis distance to detect OOD samples.
• For few-shot OOD, estimate in-distribution and OOD statistics with shrunk covariance and data augmentation, using a difference of Mahalanobis terms.
• Optionally incorporate labels via supervised contrastive loss (SSD+) to achieve tuning-parameter-free, state-of-the-art detection.
• Evaluate using AUROC, FPR at TPR=95%, and AUPR across CIFAR-10/100, STL-10, ImageNet, and other datasets.

Experimental results

Research questions

• RQ1 Can an outlier detector trained only on unlabeled in-distribution data match or surpass supervised detectors on image OOD tasks?
• RQ2 How effective is a cluster-conditioned Mahalanobis detector when using self-supervised representations for OOD detection?
• RQ3 Do few-shot OOD scenarios benefit from statistics-based adaptations with covariance shrinkage and data augmentation?
• RQ4 Does incorporating labels through supervised contrastive losses improve OOD detection performance without tuning parameters?

Key findings

• SSD significantly outperforms most unsupervised outlier detectors by large margins on standard image datasets.
• SSD achieves performance on par with, and sometimes better than, supervised detectors that use labeled in-distribution data.
• Few-shot OOD extension (SSD k) yields notable gains with access to a small number of targeted OOD samples and shrunk covariance with data augmentation.
• Incorporating labels via SSD+ and supervised contrastive loss provides state-of-the-art performance without extra tuning parameters.
• Across multiple dataset pairs, self-supervised representations often outperform supervised representations in OOD detection tasks (with substantial AUROC improvements in several cases).
• SSD+ with five-shot OOD and label incorporation can surpass prior supervised methods on several benchmarks (e.g., CIFAR-100 vs CIFAR-10, etc.).