[Paper Review] Deep Evidential Learning for Bayesian Quantile Regression

Frederik Boe Hüttel, Filipe Rodrigues+1 more|arXiv (Cornell University)|Aug 21, 2023

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper proposes a deep Bayesian quantile regression model using evidential learning to estimate both aleatoric and epistemic uncertainty in a single forward pass, without assuming Gaussian noise. It enables flexible, non-Gaussian uncertainty modeling by predicting quantiles directly, achieving calibrated uncertainty estimates and robustness to out-of-distribution data.

ABSTRACT

It is desirable to have accurate uncertainty estimation from a single deterministic forward-pass model, as traditional methods for uncertainty quantification are computationally expensive. However, this is difficult because single forward-pass models do not sample weights during inference and often make assumptions about the target distribution, such as assuming it is Gaussian. This can be restrictive in regression tasks, where the mean and standard deviation are inadequate to model the target distribution accurately. This paper proposes a deep Bayesian quantile regression model that can estimate the quantiles of a continuous target distribution without the Gaussian assumption. The proposed method is based on evidential learning, which allows the model to capture aleatoric and epistemic uncertainty with a single deterministic forward-pass model. This makes the method efficient and scalable to large models and datasets. We demonstrate that the proposed method achieves calibrated uncertainties on non-Gaussian distributions, disentanglement of aleatoric and epistemic uncertainty, and robustness to out-of-distribution samples.

Motivation & Objective

• Address the limitation of existing evidential regression models that assume Gaussian aleatoric uncertainty, restricting their ability to model complex, non-Gaussian noise distributions.
• Enable accurate uncertainty quantification in regression tasks where mean and variance are insufficient descriptors of the target distribution.
• Develop a method that disentangles aleatoric and epistemic uncertainty without requiring sampling or multiple forward passes.
• Ensure scalability and efficiency for large models and datasets by avoiding computationally expensive inference techniques like Monte Carlo dropout or deep ensembles.
• Demonstrate robustness to out-of-distribution samples and calibration on real-world and synthetic non-Gaussian data.

Proposed method

• Formulate a likelihood function based on the Asymmetric Laplace distribution to model quantiles in Bayesian quantile regression.
• Apply evidential learning by placing a Dirichlet prior over the parameters of the likelihood function, enabling the network to output evidence for the quantile parameters.
• Train a deep neural network to predict the parameters of the evidential distribution (i.e., evidence for quantiles), allowing uncertainty estimation via closed-form expressions.
• Use a proper scoring rule (e.g., quantile loss) as the training objective to ensure well-calibrated quantile predictions.
• Decouple aleatoric uncertainty (via quantile predictions) from epistemic uncertainty (via evidence strength) in a single forward pass.
• Leverage the evidential framework to estimate epistemic uncertainty as a function of evidence concentration, avoiding sampling or ensemble methods.

Experimental results

Research questions

• RQ1 Can evidential learning be extended to model non-Gaussian aleatoric uncertainty in regression by predicting quantiles instead of mean and variance?
• RQ2 Does the proposed method achieve well-calibrated uncertainty estimates on non-Gaussian noise distributions without requiring sampling during inference?
• RQ3 Can the model effectively disentangle aleatoric and epistemic uncertainty in a single forward pass, even under distributional shift?
• RQ4 How does the proposed model compare in performance and efficiency to state-of-the-art Bayesian neural network methods on real-world regression tasks?
• RQ5 To what extent can evidential quantile regression detect out-of-distribution samples using uncertainty estimates?

Key findings

• The proposed model achieves well-calibrated prediction intervals on non-Gaussian distributions, such as exponential and asymmetric noise, outperforming mean-variance-based evidential models.
• The method successfully disentangles aleatoric and epistemic uncertainty, with epistemic uncertainty increasing near distributional boundaries and out-of-distribution regions.
• The model demonstrates robustness to out-of-distribution samples, with epistemic uncertainty reliably increasing on such inputs.
• The approach scales efficiently to large-scale computer vision tasks, as shown in experiments on the Apolloscape dataset, where evidential uncertainty captures object boundaries.
• Despite not outperforming baselines in predictive accuracy, the method achieves competitive performance with significantly reduced inference cost due to single-pass inference.
• The model's uncertainty estimates are more flexible than Gaussian-based evidential regression, as it can model asymmetric and heavy-tailed distributions without distributional assumptions.