[Paper Review] Are Bayesian neural networks intrinsically good at out-of-distribution detection?

Christian Henning, Francesco D’Angelo+1 more|arXiv (Cornell University)|Jul 26, 2021

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper challenges the assumption that Bayesian neural networks (BNNs) are inherently good at out-of-distribution (OOD) detection. Using exact inference in infinite-width networks and Hamiltonian Monte Carlo (HMC) in finite-width networks, it shows that common architectures induce kernels—especially with standard weight priors—that fail to produce meaningful epistemic uncertainty for OOD inputs, undermining their reliability as OOD detectors.

ABSTRACT

The need to avoid confident predictions on unfamiliar data has sparked interest in out-of-distribution (OOD) detection. It is widely assumed that Bayesian neural networks (BNN) are well suited for this task, as the endowed epistemic uncertainty should lead to disagreement in predictions on outliers. In this paper, we question this assumption and provide empirical evidence that proper Bayesian inference with common neural network architectures does not necessarily lead to good OOD detection. To circumvent the use of approximate inference, we start by studying the infinite-width case, where Bayesian inference can be exact considering the corresponding Gaussian process. Strikingly, the kernels induced under common architectural choices lead to uncertainties that do not reflect the underlying data generating process and are therefore unsuited for OOD detection. Finally, we study finite-width networks using HMC, and observe OOD behavior that is consistent with the infinite-width case. Overall, our study discloses fundamental problems when naively using BNNs for OOD detection and opens interesting avenues for future research.

Motivation & Objective

• To investigate whether Bayesian neural networks (BNNs) are intrinsically capable of out-of-distribution (OOD) detection via epistemic uncertainty.
• To challenge the widespread assumption that BNNs naturally detect OOD inputs due to uncertainty modeling.
• To analyze the kernel properties induced by common neural network architectures in the infinite-width limit and their implications for OOD detection.
• To evaluate OOD performance using exact inference (HMC) in finite-width networks and compare it to infinite-width behavior.
• To highlight the critical role of weight-space priors and the confounding effect of aleatoric uncertainty on predictive entropy.

Proposed method

• The study analyzes the infinite-width limit of fully connected neural networks using Gaussian process (GP) equivalence, deriving the induced kernel from the network's architecture and weight priors.
• It examines the kernel structure of standard architectures (e.g., ReLU, RBF) and shows that their uncertainty does not reflect the data-generating distribution, especially in OOD regions.
• The paper uses Hamiltonian Monte Carlo (HMC) for exact Bayesian inference in finite-width networks to validate findings from the infinite-width analysis.
• It introduces a reverse sampling approach using epistemic uncertainty as an energy function to generate in-distribution-like samples and validate OOD detection consistency.
• It quantitatively compares generated samples with training data using discrepancy measures (e.g., MMD) to assess whether uncertainty aligns with the true data distribution.
• It evaluates OOD detection performance using predictive entropy and identifies its confounding by aleatoric uncertainty, especially in high-dimensional settings.

Experimental results

Research questions

• RQ1 Do Bayesian neural networks with standard architectures and priors produce epistemic uncertainty that reliably identifies out-of-distribution inputs?
• RQ2 How do the kernel functions induced by common neural network architectures in the infinite-width limit affect OOD detection performance?
• RQ3 Is the OOD behavior observed in finite-width BNNs via approximate inference consistent with exact inference in the infinite-width limit?
• RQ4 To what extent does the choice of weight-space prior influence the OOD detection capability of BNNs?
• RQ5 Can epistemic uncertainty be used to generate in-distribution samples, and does this provide a reliable validation method for OOD detection?

Key findings

• The infinite-width limit of standard fully connected networks with ReLU or RBF activations induces kernels that do not reflect the true data distribution, resulting in poor OOD detection performance.
• Even under exact Bayesian inference via HMC, finite-width BNNs with standard architectures exhibit OOD detection behavior consistent with the infinite-width case, indicating a fundamental architectural limitation.
• The RBF kernel induced by certain network parameters can support good OOD detection, but only when the weight prior is carefully tuned (e.g., large $σ_{\mu}^2$), which is not typical in practice.
• Predictive entropy, commonly used as a proxy for uncertainty, is confounded by aleatoric uncertainty and thus unreliable for OOD detection in BNNs.
• Epistemic uncertainty in standard BNNs does not reliably approximate the underlying data density $p({\bm{x}})$, undermining its use as a generative model for in-distribution sampling.
• The study demonstrates that OOD detection performance is highly sensitive to the choice of weight-space prior, with improper priors leading to misleading uncertainty estimates.