[Paper Review] Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning

Stefan Depeweg, José Miguel Hernández-Lobato+2 more|arXiv (Cornell University)|Oct 19, 2017

Machine Learning and Algorithms121 citations

TL;DR

The paper decomposes predictive uncertainty in Bayesian neural networks with latent variables into epistemic and aleatoric parts, using this decomposition to enable active learning and introduce a risk-sensitive RL criterion in continuous settings.

ABSTRACT

Bayesian neural networks with latent variables are scalable and flexible probabilistic models: They account for uncertainty in the estimation of the network weights and, by making use of latent variables, can capture complex noise patterns in the data. We show how to extract and decompose uncertainty into epistemic and aleatoric components for decision-making purposes. This allows us to successfully identify informative points for active learning of functions with heteroscedastic and bimodal noise. Using the decomposition we further define a novel risk-sensitive criterion for reinforcement learning to identify policies that balance expected cost, model-bias and noise aversion.

Motivation & Objective

• Explain how to decompose predictive uncertainty in BNNs with latent variables into epistemic and aleatoric components.
• Demonstrate data-efficient active learning in the presence of heteroscedastic and bimodal noise.
• Introduce a risk-sensitive model-based RL criterion balancing model bias and noise.
• Show empirical benefits of uncertainty decomposition over baselines in nonlinear, complex-noise settings.

Proposed method

• Model: Bayesian neural networks with latent variables (BNN+LV) that include latent inputs z and weight uncertainty q(W).
• Uncertainty decomposition: derive entropy- and variance-based splits to separate epistemic and aleatoric uncertainty in predictive p(y*|x*).
• Active learning: use information-theoretic posterior entropy reduction I(W,y*) to select informative x* where epistemic uncertainty is high.
• Risk-sensitive RL: define a criterion that balances expected cost with epistemic and aleatoric risk terms in virtual roll-outs and connect to model bias and noise aversion.
• Monte Carlo estimation: approximate entropies, variances, and risk terms by sampling W~q(W), z*, and noise, and using nearest-neighbor entropy estimators where needed.
• Comparison: benchmark against GP baselines and variational/HMC inference variants to validate uncertainty decompositions.

Experimental results

Research questions

• RQ1 How can predictive uncertainty in BNNs with latent variables be decomposed into epistemic and aleatoric components?
• RQ2 Does the uncertainty decomposition improve active learning in settings with heteroscedastic or bimodal noise?
• RQ3 Can a risk-sensitive criterion leveraging this decomposition improve policy learning in model-based RL for continuous domains?
• RQ4 How does the proposed approach compare to GP baselines and other inference methods in terms of predictive performance and uncertainty quality.

Key findings

• Epistemic uncertainty aligns with data-scarce regions and helps identify informative inputs for active learning under complex noise.
• The entropy-based decomposition reveals informative inputs in heteroscedastic and bimodal data where total uncertainty fails.
• BNN+LV with uncertainty decomposition outperforms GP baselines in active learning on heteroscedastic and wet-chicken tasks.
• In RL, the risk-sensitive criterion partitions risk into epistemic (model-bias) and aleatoric (noise) components, guiding safer and more reliable policies.
• Across experiments, using the information gain for active learning and risk-weighted rollout costs reduces model-bias and improves performance relative to baselines.