[Paper Review] A Survey of Uncertainty in Deep Neural Networks

Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi+12 more|arXiv (Cornell University)|Jul 7, 2021

Fault Detection and Control Systems242 citations

TL;DR

A comprehensive survey on sources, estimation methods, calibration, and real-world challenges of uncertainty in deep neural networks, including Bayesian, ensembles, and test-time augmentation approaches.

ABSTRACT

Due to their increasing spread, confidence in neural network predictions became more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over or under confidence. Many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identified and a variety of approaches to measure and quantify uncertainty in neural networks have been proposed. This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. A comprehensive introduction to the most crucial sources of uncertainty is given and their separation into reducible model uncertainty and not reducible data uncertainty is presented. The modeling of these uncertainties based on deterministic neural networks, Bayesian neural networks, ensemble of neural networks, and test-time data augmentation approaches is introduced and different branches of these fields as well as the latest developments are discussed. For a practical application, we discuss different measures of uncertainty, approaches for the calibration of neural networks and give an overview of existing baselines and implementations. Different examples from the wide spectrum of challenges in different fields give an idea of the needs and challenges regarding uncertainties in practical applications. Additionally, the practical limitations of current methods for mission- and safety-critical real world applications are discussed and an outlook on the next steps towards a broader usage of such methods is given.

Motivation & Objective

• Identify and categorize the sources and types of uncertainty in deep neural networks (data vs. model uncertainty).
• Survey and compare major uncertainty estimation methodologies (Bayesian NNs, ensembles, test-time augmentation, deterministic models).
• Discuss calibration of uncertainty estimates and practical benchmarks for real-world applications.
• Highlight limitations, challenges, and opportunities for future research in uncertainty quantification for DNNs.

Proposed method

• Describe four steps from data to uncertainty quantification: data acquisition, network design/training, inference, and predictive uncertainty modeling.
• Differentiate factors causing uncertainty (I: variability in real world; II: measurement noise; III: model structure errors; IV: training procedure errors; V: unknown data) and how they propagate.
• Classify predictive uncertainty into data (aleatoric) and model (epistemic) uncertainties and discuss Bayesian and distributional formulations.
• Present four uncertainty estimation paradigms: single deterministic networks, Bayesian methods, ensembles, and test-time augmentation, detailing their trade-offs.
• Provide an overview of evaluation measures, calibration techniques, and available implementations and benchmarks.

Experimental results

Research questions

• RQ1 What are the principal sources and types of uncertainty in DNN predictions?
• RQ2 What are the main approaches to modeling and quantifying predictive uncertainty in DNNs, and how do they compare?
• RQ3 How can DNN uncertainty estimates be calibrated and evaluated for reliability in practice?
• RQ4 What are the practical challenges and limitations of current uncertainty quantification methods in real-world applications?

Key findings

• Uncertainty in DNN predictions arises from data uncertainty (aleatoric) and model uncertainty (epistemic), with distinct causes and reducibility properties.
• Bayesian inference, ensembles, test-time data augmentation, and deterministic models with explicit uncertainty components are the main modeling paradigms for uncertainty estimation.
• Calibration of uncertainty estimates is essential for reliability, and several calibration techniques exist alongside common evaluation datasets and benchmarks.
• In-field applications (medical imaging, robotics, Earth observation) reveal practical challenges such as domain shifts, out-of-domain inputs, and safety-critical decision requirements.
• The paper provides a framework to map uncertainty sources to practical estimation methods and highlights when each approach is preferable under resource constraints.