[Paper Review] Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks

Agustinus Kristiadi, Matthias Hein+1 more|arXiv (Cornell University)|Feb 24, 2020

Adversarial Robustness in Machine LearningComputer Science52 references34 citations

TL;DR

The paper proves that Gaussian approximations to ReLU network weights, including last-layer Bayesian approaches, mitigate the overconfidence of predictions far from training data, with theory and experiments supporting a cost-efficient, “a bit Bayesian” remedy.

ABSTRACT

The point estimates of ReLU classification networks---arguably the most widely used neural network architecture---have been shown to yield arbitrarily high confidence far away from the training data. This architecture, in conjunction with a maximum a posteriori estimation scheme, is thus not calibrated nor robust. Approximate Bayesian inference has been empirically demonstrated to improve predictive uncertainty in neural networks, although the theoretical analysis of such Bayesian approximations is limited. We theoretically analyze approximate Gaussian distributions on the weights of ReLU networks and show that they fix the overconfidence problem. Furthermore, we show that even a simplistic, thus cheap, Bayesian approximation, also fixes these issues. This indicates that a sufficient condition for a calibrated uncertainty on a ReLU network is "to be a bit Bayesian". These theoretical results validate the usage of last-layer Bayesian approximation and motivate a range of a fidelity-cost trade-off. We further validate these findings empirically via various standard experiments using common deep ReLU networks and Laplace approximations.

Motivation & Objective

• Motivate and formalize the overconfidence issue in ReLU networks and its implications for calibration and robustness.
• Develop theoretical results showing that Gaussian weight approximations stabilize predictive confidence away from training data.
• Show that a minimal Bayesian treatment, especially last-layer Bayes, suffices to fix calibration without harming in-distribution accuracy.
• Validate the theory with empirical experiments on common deep ReLU architectures using Laplace/Gaussian approximations.

Proposed method

• Model the network predictions under a Gaussian approximation p(theta|D) over weights and derive predictive distributions.
• Use probit/phi approximations to relate Gaussian weight uncertainty to calibrated predictive probabilities (equations 3–5).
• Prove an invariance property showing MAP decisions remain unchanged under the Gaussian marginalization (Proposition 2.2).
• Derive asymptotic bounds on predictive confidence as input samples are scaled far from training data (Theorems 2.3 and 2.4).
• Show that last-layer Gaussian approximations suffice to achieve bounded far-away confidence (Theorem 2.4).
• Apply Laplace approximations (full and last-layer) to analyze and quantify the impact on calibration and OOD detection (Propositions 2.5 and related discussions).

Experimental results

Research questions

• RQ1 Does introducing Gaussian uncertainty over neural network weights alleviate the overconfidence of ReLU classifiers far from the training data?
• RQ2 Is a full network Bayesian treatment necessary, or can a lightweight last-layer Bayesian approach suffice to achieve calibrated uncertainty?
• RQ3 How do Gaussian approximations affect the asymptotic behavior of predictive confidence for out-of-distribution inputs?
• RQ4 Do these theoretical insights extend to multi-class classification and practical OOD detection scenarios?
• RQ5 What are practical strategies to set prior variance in Laplace approximations to balance in-distribution accuracy and out-of-distribution calibration?

Key findings

• Gaussian weight uncertainty mitigates overconfidence in ReLU networks by driving far-away predictive confidence to a finite bound determined by the mean and covariance.
• A last-layer Gaussian approximation suffices to obtain calibrated uncertainty without changing the decision boundary of the trained MAP network.
• All-layer Gaussian approximations further bound far-away confidence and relate it to the Jacobian and the smallest eigenvalue of the weight covariance.
• Laplace approximations provide practical means to obtain the Bayesian corrections without re-training, and the prior variance controls far-away confidence.
• Empirical results on common architectures (LeNet, ResNet-18) and datasets (binary and multi-class) corroborate theoretical predictions and improve OOD detection performance.
• Last-layer Bayesian methods often outperform full Bayesian approaches in practice for this calibration task.