[Paper Review] A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Dan Hendrycks, Kevin Gimpel|arXiv (Cornell University)|Oct 7, 2016

Neural Networks and Applications1,615 citations

TL;DR

The paper introduces a simple baseline that uses maximum softmax probabilities to detect misclassified and out-of-distribution examples across vision, NLP, and speech tasks, and also proposes an auxiliary abnormality detector that can outperform the baseline on some tasks.

ABSTRACT

We consider the two related problems of detecting if an example is misclassified or out-of-distribution. We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection. We assess performance by defining several tasks in computer vision, natural language processing, and automatic speech recognition, showing the effectiveness of this baseline across all. We then show the baseline can sometimes be surpassed, demonstrating the room for future research on these underexplored detection tasks.

Motivation & Objective

• Motivate and quantify the need to detect when a model will misclassify or when an input is out-of-distribution (OOD).
• Propose a simple baseline using softmax maximum probabilities to detect errors and OOD inputs across multiple domains.
• Explore enhancements via auxiliary input reconstruction to improve abnormality detection beyond the baseline.
• Provide standardized tasks and metrics for evaluating error and OOD detection in neural networks.

Proposed method

• Use the maximum softmax probability from trained classifiers as a detection score for misclassification and for in-/out-of-distribution discrimination.
• Evaluate the baseline across diverse tasks and datasets in computer vision, NLP, and automatic speech recognition, reporting AUROC and AUPR metrics.
• Investigate a reconstruction-based auxiliary model (an input-reconstruction decoder plus an abnormality module) to improve detection of abnormal inputs.
• Demonstrate that softmax-based scores are not direct confidence estimates but can effectively separate correct vs incorrect and in-vs-OOD examples.
• Introduce standard tasks and evaluation metrics for error and OOD detection to guide future work.

Experimental results

Research questions

• RQ1 Can a simple baseline based on softmax maximum probabilities reliably detect misclassified examples across diverse domains (vision, NLP, speech)?
• RQ2 Can the same baseline distinguish in-distribution from out-of-distribution inputs effectively?
• RQ3 Does an auxiliary reconstruction-based abnormality detector improve detection performance beyond the softmax baseline in specific settings?
• RQ4 What standard tasks and metrics can be used to evaluate error and out-of-distribution detection in neural networks?

Key findings

• Softmax maximum probability serves as a surprisingly effective baseline for detecting misclassifications across MNIST, CIFAR-10, CIFAR-100, and NLP/speech tasks.
• The baseline also enables discrimination between in-distribution and out-of-distribution data across various datasets (e.g., SUN, Omniglot, notMNIST, Gaussian noise).
• In several tasks, AUC-ROC and AUPR indicate strong detection performance, though the baseline is not universally optimal, leaving room for improvement.
• An auxiliary abnormality module with an input-reconstruction decoder can surpass the baseline in some scenarios (e.g., TIMIT, MNIST), suggesting additional information in internal representations for abnormality detection.
• The work demonstrates robust cross-domain applicability and provides a foundation for future research in error and OOD detection.