[Paper Review] Learning Confidence for Out-of-Distribution Detection in Neural Networks

Terrance DeVries, Graham W. Taylor|arXiv (Cornell University)|Feb 13, 2018

Adversarial Robustness in Machine LearningComputer Science5 references356 citations

TL;DR

The paper introduces a confidence-estimation branch in neural networks to detect out-of-distribution inputs, improving over softmax-based methods and ODIN without requiring out-of-distribution labels. Confidence misalignment is mitigated via a loss that balances task performance with calibrated confidence.

ABSTRACT

Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safely perform. To jointly address these issues, we propose a method of learning confidence estimates for neural networks that is simple to implement and produces intuitively interpretable outputs. We demonstrate that on the task of out-of-distribution detection, our technique surpasses recently proposed techniques which construct confidence based on the network's output distribution, without requiring any additional labels or access to out-of-distribution examples. Additionally, we address the problem of calibrating out-of-distribution detectors, where we demonstrate that misclassified in-distribution examples can be used as a proxy for out-of-distribution examples.

Motivation & Objective

• Motivate calibrated confidence estimates to identify when predictions may be wrong and inputs are out-of-distribution.
• Propose a trainable confidence branch that augments standard classifiers.
• Show that confidence-based thresholding improves OOD detection across architectures and datasets.
• Demonstrate that misclassified in-distribution examples can calibrate OOD detectors without OOD data.

Proposed method

• Add a confidence estimation branch in parallel to the prediction branch that outputs a scalar c in [0,1].
• Modify prediction probabilities by p' = c * p + (1 - c) * y, where y is the target distribution.
• Train with a task loss L_t (e.g., negative log likelihood) plus a confidence loss L_c = -log(c) weighted by lambda: L = L_t + lambda * L_c.
• Introduce a budget parameter beta to keep L_c around beta by adjusting lambda after each update.
• Use data augmentation and misclassified example retention (e.g., Cutout) to prevent overfitting and keep useful misclassifications for learning confidence.
• Apply input preprocessing to boost separation between in- and out-of-distribution by pushing inputs toward higher confidence using perturbed inputs derived from the gradient of L_c.

Experimental results

Research questions

• RQ1 Can a learned confidence estimate per input meaningfully separate in-distribution from out-of-distribution data without OOD labels?
• RQ2 Does confidence-based thresholding outperform softmax-based thresholding and ODIN across multiple architectures and OOD datasets?
• RQ3 Is misclassification in-distribution usable as a proxy to calibrate OOD detectors when OOD samples are unavailable?
• RQ4 How should training hyperparameters (lambda, beta) be set to preserve classification accuracy while maintaining useful confidence estimates?

Key findings

• Confidence estimates enable better separation of in- and out-of-distribution examples than softmax-based methods on most architectures and datasets.
• Thresholding on learned confidence with input preprocessing (and temperature scaling) outperforms ODIN in most experiments.
• Model training with confidence branches preserves in-distribution accuracy comparable to baseline models.
• Misclassified in-distribution examples can serve as a practical proxy for calibrating OOD detectors when true OOD data are unavailable.