[Paper Review] Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Need

Jingyao Li, Pengguang Chen+4 more|arXiv (Cornell University)|Feb 6, 2023

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes MOOD, a novel out-of-distribution (OOD) detection framework that leverages masked image modeling (MIM) as a self-supervised pretext task to learn robust in-distribution (ID) representations without using any OOD samples. By focusing on reconstruction-based learning, MOOD achieves state-of-the-art performance across multiple OOD detection benchmarks, outperforming prior SOTA methods by up to 5.7% in AUROC, even surpassing models that use 10-shot outlier exposure.

ABSTRACT

The core of out-of-distribution (OOD) detection is to learn the in-distribution (ID) representation, which is distinguishable from OOD samples. Previous work applied recognition-based methods to learn the ID features, which tend to learn shortcuts instead of comprehensive representations. In this work, we find surprisingly that simply using reconstruction-based methods could boost the performance of OOD detection significantly. We deeply explore the main contributors of OOD detection and find that reconstruction-based pretext tasks have the potential to provide a generally applicable and efficacious prior, which benefits the model in learning intrinsic data distributions of the ID dataset. Specifically, we take Masked Image Modeling as a pretext task for our OOD detection framework (MOOD). Without bells and whistles, MOOD outperforms previous SOTA of one-class OOD detection by 5.7%, multi-class OOD detection by 3.0%, and near-distribution OOD detection by 2.1%. It even defeats the 10-shot-per-class outlier exposure OOD detection, although we do not include any OOD samples for our detection

Motivation & Objective

• To investigate whether reconstruction-based self-supervised learning can outperform classification-based approaches in OOD detection.
• To challenge the assumption that OOD sample exposure is necessary for high-performance OOD detection.
• To demonstrate that learning intrinsic data distributions via MIM leads to more robust ID representation learning.
• To establish a simple, effective OOD detection framework that does not rely on OOD data or complex architectures.

Proposed method

• The method employs masked image modeling (MIM) as a self-supervised pretext task, where random patches of input images are masked and the model is trained to reconstruct them.
• The model uses discrete VAE tokens as supervision signals to reconstruct the masked image patches, forcing it to learn pixel-level data distributions.
• A vision transformer backbone processes the corrupted image input and predicts the missing patches, learning generalizable ID representations in the process.
• The final OOD detection is performed using Mahalanobis distance between test features and the ID training feature distribution.
• No OOD samples are used during training or inference, making the method purely ID-focused.
• The framework is evaluated across one-class, multi-class, near-distribution, and few-shot outlier exposure OOD detection tasks.

Experimental results

Research questions

• RQ1 Can reconstruction-based self-supervised learning outperform classification-based representation learning for OOD detection?
• RQ2 Does learning the intrinsic data distribution via MIM lead to better generalization for OOD detection than shortcut learning in classification tasks?
• RQ3 Is it possible to achieve SOTA OOD detection performance without any exposure to OOD samples?
• RQ4 How does MIM-based representation learning compare to state-of-the-art methods that use outlier exposure?

Key findings

• MOOD achieves 94.9% AUROC on one-class OOD detection, a 5.7% improvement over the previous SOTA (CSI) on CIFAR-10.
• On multi-class OOD detection, MOOD reaches 97.6% AUROC, a 3.0% gain over the current SOTA (SSD+) on CIFAR-10.
• For near-distribution OOD detection, MOOD achieves 98.3% AUROC, a 2.1% improvement over the SOTA (R50+ViT) on CIFAR-10-CIFAR100 pairs.
• In few-shot outlier exposure OOD detection, MOOD achieves 99.41% AUROC without using any OOD samples, surpassing the SOTA (R50+ViT) at 99.29% with 10 OOD samples per class.
• MOOD reduces the number of mistakenly classified OOD samples by 79% on average in multi-class detection, compared to the SOTA (SSD+) at 95% TPR.
• The results demonstrate that reconstruction-based self-supervision provides a more effective inductive bias for OOD detection than classification-based or contrastive learning approaches.