[Paper Review] Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolation

Jianing Zhu, Yu Geng+5 more|arXiv (Cornell University)|Oct 21, 2023

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes Diversified Outlier Exposure (DivOE), a novel framework that enhances out-of-distribution (OOD) detection by synthesizing more informative outliers through multi-step optimization and informative extrapolation. By diversifying the auxiliary outlier distribution beyond the original data, DivOE improves OOD generalization and achieves state-of-the-art performance with lower FPR95 across multiple benchmarks, including CIFAR-10, CIFAR-100, and ImageNet.

ABSTRACT

Out-of-distribution (OOD) detection is important for deploying reliable machine learning models on real-world applications. Recent advances in outlier exposure have shown promising results on OOD detection via fine-tuning model with informatively sampled auxiliary outliers. However, previous methods assume that the collected outliers can be sufficiently large and representative to cover the boundary between ID and OOD data, which might be impractical and challenging. In this work, we propose a novel framework, namely, Diversified Outlier Exposure (DivOE), for effective OOD detection via informative extrapolation based on the given auxiliary outliers. Specifically, DivOE introduces a new learning objective, which diversifies the auxiliary distribution by explicitly synthesizing more informative outliers for extrapolation during training. It leverages a multi-step optimization method to generate novel outliers beyond the original ones, which is compatible with many variants of outlier exposure. Extensive experiments and analyses have been conducted to characterize and demonstrate the effectiveness of the proposed DivOE. The code is publicly available at: https://github.com/tmlr-group/DivOE.

Motivation & Objective

• Address the critical challenge that existing outlier exposure (OE) methods rely on limited, non-representative auxiliary outliers, which may not cover the boundary between in-distribution and OOD data.
• Investigate how to improve OOD detection performance when the given auxiliary outliers are not sufficiently informative or diverse.
• Develop a framework that actively expands the surrogate OOD distribution by generating more informative, extrapolated outliers to bridge the distribution gap.
• Ensure compatibility with various OE variants and backbone architectures to maximize practical utility.
• Demonstrate consistent performance gains across diverse OOD detection benchmarks, especially under low-data regimes for auxiliary outliers.

Proposed method

• Propose a new learning objective that explicitly maximizes the divergence between original outliers and synthesized, extrapolated outliers to promote diversity and informativeness.
• Implement a multi-step optimization process to generate novel outliers by perturbing original auxiliary outliers in high-impact directions, guided by model uncertainty and gradient signals.
• Integrate the extrapolated outliers into the training process via standard OE fine-tuning, enabling the model to learn broader OOD boundaries.
• Leverage mixup-style data augmentation in conjunction with extrapolation to further improve generalization and robustness.
• Use energy-based scoring and loss difference analysis to validate the informativeness of generated outliers, ensuring they lie in critical decision regions.
• Design the framework to be compatible with multiple OOD detection methods (e.g., Energy, NTOM, POEM, OE) and backbone networks.

Experimental results

Research questions

• RQ1 Can we improve OOD detection performance when the auxiliary outliers are insufficiently representative or sparse near the decision boundary?
• RQ2 How can we systematically generate more informative outliers that better generalize to unseen OOD inputs?
• RQ3 Does extrapolating beyond the original auxiliary outlier distribution lead to a broader and more robust OOD-robust decision boundary?
• RQ4 How does the performance of DivOE compare to standard OE and other SOTA methods under varying numbers of auxiliary outliers?
• RQ5 What is the impact of different perturbation magnitudes (ε) on the quality and informativeness of the generated outliers?

Key findings

• DivOE achieves state-of-the-art performance on CIFAR-10 and CIFAR-100 benchmarks, reducing the average FPR95 to 4.97% when combined with OE and mixup.
• On CIFAR-100, DivOE+Mixup achieves an FPR95 of 4.97% and AUROC of 76.06%, outperforming standard OE and other baselines.
• With only 1,000 auxiliary outliers, DivOE maintains strong performance (FPR95 ≈ 48.35%) compared to standard OE (FPR95 ≈ 54.31%), demonstrating robustness under low-data regimes.
• The OOD loss difference between original and extrapolated outliers increases with ε, peaking at ε=0.2 (20.29) and reaching 24.16 at ε=0.5, confirming the model learns from increasingly informative samples.
• Visualizations show that extrapolated outliers cover previously unexplored regions of the feature space, particularly near the decision boundary.
• The method is effective across multiple OOD detection heads (e.g., Energy, NTOM, POEM), indicating broad compatibility and generalization.