[论文解读] Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output

Vahdat Abdelzad, Krzysztof Czarnecki+4 more|arXiv (Cornell University)|Oct 23, 2019

Anomaly Detection Techniques and Applications参考文献 37被引用 35

一句话总结

本文提出一种 OOD 探测器，使用在预训练分类器的早期（最佳 OOD 识别层）特征上的一类分类器，避免使用 OOD 数据或重新训练，且可选的输入预处理以提升性能。

ABSTRACT

Deep neural networks achieve superior performance in challenging tasks such as image classification. However, deep classifiers tend to incorrectly classify out-of-distribution (OOD) inputs, which are inputs that do not belong to the classifier training distribution. Several approaches have been proposed to detect OOD inputs, but the detection task is still an ongoing challenge. In this paper, we propose a new OOD detection approach that can be easily applied to an existing classifier and does not need to have access to OOD samples. The detector is a one-class classifier trained on the output of an early layer of the original classifier fed with its original training set. We apply our approach to several low- and high-dimensional datasets and compare it to the state-of-the-art detection approaches. Our approach achieves substantially better results over multiple metrics.

研究动机与目标

• Motivate the need to detect out-of-distribution inputs in DNNs for safety-critical tasks.
• Propose a detector that does not require retraining the classifier or access to OOD samples.
• Identify and leverage an optimal early-layer latent space (OODL) where ID and OOD distributions are well separated.
• Evaluate the method across multiple low- and high-dimensional datasets against state-of-the-art detectors.

提出的方法

• Define a latent space via an early layer (OODL) whose features separate ID and OOD distributions.
• Search for the OODL by evaluating detection error across layers using a one-class SVM (OSVM) trained on training data features.
• Use a one-class detector on the OODL features to distinguish ID vs. OOD inputs at inference.
• For convolutional layers, reduce feature map dimensionality by channel-wise mean pooling before OSVM.
• Optionally apply input preprocessing (as in ODIN) to perturb inputs and improve detection, though the method already performs well without it.

实验结果

研究问题

• RQ1 Can an early layer of a fixed classifier provide a latent space where ID and OOD distributions are separable enough for detection without OOD samples?
• RQ2 Which layer in a given network serves as the optimal OOD discernment layer (OODL) for robust OOD detection?
• RQ3 Does a one-class detector on OODL features outperform existing max-softmax, ODIN, and MD-based detectors across datasets?
• RQ4 What is the impact of input preprocessing on the proposed method’s performance?

主要发现

• The proposed approach yields substantially better detection performance than baseline and state-of-the-art methods on multiple metrics.
• The optimal OOD discernment layer (OODL) is typically a low-level layer, consistent across different OOD datasets but varying with ID dataset and architecture.
• Using OSVM on reduced-dimension early-layer features enables effective high-dimensional OOD detection without requiring OOD data during training.
• Input preprocessing can further enhance detection, but the method already achieves strong results without it.
• The detector is applicable to existing pretrained classifiers without retraining and without access to OOD samples.