[Paper Review] Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications

David Ruhe, Giovanni Cinà+3 more|arXiv (Cornell University)|Jun 20, 2019

Machine Learning in Healthcare17 references4 citations

TL;DR

This paper proposes using Bayesian Neural Networks (BNNs) in intensive care units (ICUs) to improve trustworthiness in medical machine learning by quantifying predictive uncertainty. By deriving analytical bounds on prediction loss and applying BNNs to the MIMIC-III dataset, the authors show that uncertainty reliably identifies out-of-domain patients—such as newborns and ethnic minorities—increasing uncertainty by 230% and 130%, respectively—thereby reducing risk of erroneous decisions.

ABSTRACT

The Intensive Care Unit (ICU) is a hospital department where machine learning has the potential to provide valuable assistance in clinical decision making. Classical machine learning models usually only provide point-estimates and no uncertainty of predictions. In practice, uncertain predictions should be presented to doctors with extra care in order to prevent potentially catastrophic treatment decisions. In this work we show how Bayesian modelling and the predictive uncertainty that it provides can be used to mitigate risk of misguided prediction and to detect out-of-domain examples in a medical setting. We derive analytically a bound on the prediction loss with respect to predictive uncertainty. The bound shows that uncertainty can mitigate loss. Furthermore, we apply a Bayesian Neural Network to the MIMIC-III dataset, predicting risk of mortality of ICU patients. Our empirical results show that uncertainty can indeed prevent potential errors and reliably identifies out-of-domain patients. These results suggest that Bayesian predictive uncertainty can greatly improve trustworthiness of machine learning models in high-risk settings such as the ICU.

Motivation & Objective

• To enhance trustworthiness of machine learning in high-stakes ICU settings by incorporating predictive uncertainty.
• To address the risk of catastrophic errors from out-of-distribution predictions in deterministic models.
• To demonstrate that Bayesian uncertainty can detect patients with rare or unseen combinations of symptoms and vital signs.
• To provide theoretical bounds linking uncertainty to prediction loss, showing uncertainty mitigates risk.
• To evaluate BNNs using Bayes-by-Backprop on real-world ICU data, comparing performance to deterministic models.

Proposed method

• The authors use Bayesian Neural Networks (BNNs) with variational inference to approximate the true posterior over model weights, enabling uncertainty quantification.
• They derive an analytical bound on cross-entropy loss in terms of predictive uncertainty, showing that higher uncertainty correlates with lower risk of high-loss predictions.
• Bayes-by-Backprop is employed to directly estimate the posterior distribution over weights, avoiding the need for hyperparameter tuning of dropout rates as in MC Dropout.
• The model is trained on the MIMIC-III dataset to predict ICU patient mortality using clinical and lab data.
• Out-of-domain detection is evaluated by testing on newborns and ethnic minorities excluded from training, measuring uncertainty increases.
• The performance is compared to a deterministic random forest model, with cumulative loss and uncertainty trends analyzed across prediction confidence levels.

Experimental results

Research questions

• RQ1 Can predictive uncertainty derived from Bayesian Neural Networks be used to bound and reduce prediction loss in ICU mortality prediction?
• RQ2 How well can BNNs detect out-of-domain patients, such as newborns or underrepresented ethnic groups, through increased uncertainty?
• RQ3 Does the use of Bayes-by-Backprop provide more reliable uncertainty estimates than MC Dropout in clinical deep learning settings?
• RQ4 Is there a measurable relationship between prediction uncertainty and the risk of high-loss errors in real ICU data?
• RQ5 Can BNNs serve as a safety mechanism in clinical decision support by flagging rare or novel patient profiles?

Key findings

• Theoretical bounds show that predictive uncertainty can mitigate prediction loss, with higher uncertainty associated with lower risk of high-loss errors.
• On the MIMIC-III dataset, test set prediction loss increased superlinearly with rising uncertainty, confirming that uncertainty correlates with reduced risk.
• When presented with newborn patients, the BNN increased its predictive uncertainty by 230% on average, indicating detection of out-of-domain cases.
• For an underrepresented ethnic minority group, the BNN showed a 130% increase in uncertainty (p < .0001), demonstrating effective detection of rare patient profiles.
• The BNN outperformed a deterministic random forest in terms of uncertainty calibration, with uncertainty reliably signaling unreliable predictions.
• The results suggest that BNNs can be deployed not only for classification but primarily as a safety mechanism through uncertainty output, even when accuracy is slightly lower than non-Bayesian models.