[Paper Review] Provably Robust Detection of Out-of-distribution Data (almost) for free

Alexander Meinke, Julian Bitterwolf+1 more|arXiv (Cornell University)|Jun 8, 2021

Adversarial Robustness in Machine Learning50 references4 citations

TL;DR

This paper proposes ProoD, a method for provably adversarially robust out-of-distribution (OOD) detection that achieves high clean accuracy and strong OOD detection performance without sacrificing model reliability. By modifying the classifier's architecture to include a learnable bias shift and applying interval bound propagation (IBP) during training, ProoD provides mathematically guaranteed upper bounds on confidence for OOD samples under $l_∞$-perturbations, ensuring robustness without degrading in-distribution accuracy.

ABSTRACT

The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even if trained to be non-confident on OOD data, one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples. We show that two previously published defenses can be broken by better adapted attacks, highlighting the importance of robustness guarantees around OOD data. Since the existing method for this task is hard to train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably adversarially robust OOD detection and high clean accuracy. Moreover, by slightly modifying the classifier's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks. We provide code for all our experiments.

Motivation & Objective

• To address the vulnerability of OOD detection methods to adversarial perturbations that can artificially increase classifier confidence on out-of-distribution inputs.
• To develop a method that provides provable guarantees on OOD confidence under $l_\infty$-bounded adversarial attacks, ensuring robustness without compromising clean accuracy.
• To eliminate the asymptotic overconfidence problem of standard neural networks far from the training data distribution.
• To combine high in-distribution accuracy with certified robustness for OOD detection, enabling practical deployment in safety-critical systems.

Proposed method

• The method modifies the classifier's architecture by introducing a learnable bias shift $\Delta$ to prevent asymptotic overconfidence.
• It applies interval bound propagation (IBP) during training to compute certified upper bounds on the confidence of OOD samples within an $l_\infty$-neighborhood.
• The training objective minimizes the certified upper bound on confidence for out-of-distribution data, ensuring robustness under adversarial perturbations.
• The approach is compatible with standard OOD detection baselines and can be combined with already-trained, robust models without retraining.
• The method uses a binary discriminator head to classify inputs as in- or out-of-distribution based on confidence thresholds.
• It provides pointwise certificates for each OOD sample, mathematically guaranteeing that no $l_\infty$-perturbation can raise confidence above the certified bound.

Experimental results

Research questions

• RQ1 Can OOD detection be made provably robust against $l_\infty$-adversarial attacks while maintaining high clean accuracy?
• RQ2 Can the asymptotic overconfidence of standard neural networks on OOD data be provably eliminated through architectural and training modifications?
• RQ3 Can certified robustness for OOD detection be achieved without retraining or degrading performance on in-distribution data?
• RQ4 How does the proposed method compare empirically and theoretically to existing OOD detection and adversarially robust methods?
• RQ5 Can the method be composed with already-trained, in-distribution robust models to enhance OOD robustness without performance loss?

Key findings

• ProoD achieves high clean accuracy (95.6% on CIFAR-10) comparable to state-of-the-art methods like OE, while providing certified robustness.
• On the CIFAR-100 out-of-distribution benchmark, ProoD with $\Delta=3$ achieves a certified GAUC of 48.6% under $\epsilon=0.01$ $l_\infty$-perturbations, significantly outperforming non-robust baselines.
• The method provides non-zero GAUCs and certified confidence bounds, even for OOD data far from the training distribution, unlike standard models that become asymptotically overconfident.
• When combined with a robust ResNet-18 from RobustBench, ProoD improves or maintains OOD detection performance across all datasets, with no drop in clean or robust accuracy.
• Error bars across five runs show minimal variance, confirming the stability of the method and the consistency of its performance.
• ProoD-Disc, a variant with a discrete bias shift, achieves 67.7% AUC and 61.6% GAUC on CIFAR-100, demonstrating strong empirical robustness.