[Paper Review] Robust Out-of-distribution Detection for Neural Networks

Jiefeng Chen, Yixuan Li+3 more|arXiv (Cornell University)|Mar 21, 2020

Adversarial Robustness in Machine LearningComputer Science50 references31 citations

TL;DR

The paper shows that modern OOD detectors are brittle under small adversarial perturbations on both in-distribution and out-of-distribution inputs, and introduces ALOE, a robust adversarial training method to improve OOD detection.

ABSTRACT

Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in the real world. Existing approaches for detecting OOD examples work well when evaluated on benign in-distribution and OOD samples. However, in this paper, we show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs with minimal adversarial perturbations which don't change their semantics. Formally, we extensively study the problem of Robust Out-of-Distribution Detection on common OOD detection approaches, and show that state-of-the-art OOD detectors can be easily fooled by adding small perturbations to the in-distribution and OOD inputs. To counteract these threats, we propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples. Our method can be flexibly combined with, and render existing methods robust. On common benchmark datasets, we show that ALOE substantially improves the robustness of state-of-the-art OOD detection, with 58.4% AUROC improvement on CIFAR-10 and 46.59% improvement on CIFAR-100.

Motivation & Objective

• Motivate robust OOD detection for open-world deployment where inputs can be adversarially perturbed.
• Show brittleness of existing OOD detectors under small semantic-preserving perturbations.
• Propose ALOE to robustify OOD detectors via adversarial training with inliers and outliers.
• Offer empirical evidence that ALOE substantially improves robustness while preserving classification accuracy.
• Provide a reusable code base to facilitate reproducibility and further research.

Proposed method

• Formally define robust OOD detection and adversarial perturbations within an epsilon-ball around inputs.
• Propose ALOE: adversarial training that includes in-distribution perturbations maximizing negative log-likelihood and outlier perturbations maximizing KL-divergence to a uniform distribution.
• Frame training as a min-max optimization solved with projected gradient descent (PGD).
• Allow integration of ALOE with existing detectors (e.g., MSP, ODIN) to enhance robustness.
• Discuss variants ADV and AOE that focus on adversarial inliers or combined inliers/outliers.
• Release accompanying code to support reproducibility.

Experimental results

Research questions

• RQ1 Do existing OOD detectors fail under small, semantics-preserving adversarial perturbations on in-distribution and out-of-distribution inputs?
• RQ2 Can a robust training paradigm (ALOE) substantially improve OOD detection performance under such attacks?
• RQ3 How does ALOE interact with and potentially improve other detectors like ODIN?
• RQ4 What is the impact of adversarial training on clean (unattacked) classification accuracy?
• RQ5 What insights explain why adversarial perturbations targeting classifiers should be treated as in-distribution for OOD objectives?

Key findings

• Classic OOD detectors (MSP, ODIN, Mahalanobis, OE) degrade dramatically under adversarial perturbations (epsilon = 1/255, m = 10).
• ALOE substantially improves robust OOD detection, e.g., AUROC improvements up to 58.4% on CIFAR-10 and 46.59% on CIFAR-100 under attack (relative to prior methods).
• ALOE can be combined with ODIN for further performance gains and often maintains or closely matches clean accuracy.
• Adversarial examples aimed at the image classifier f(x) with small perturbations tend to be in-distribution rather than OOD under their robust framework.
• A robust training objective that uses inlier perturbations and outlier exposure via KL-divergence-based perturbations yields the strongest robustness gains among evaluated variants.