[Paper Review] Getting a CLUE: A Method for Explaining Uncertainty Estimates

Javier Antorán, Umang Bhatt+3 more|arXiv (Cornell University)|Jun 11, 2020

Explainable Artificial Intelligence (XAI)50 references41 citations

TL;DR

CLUE explains neural network predictive uncertainty by producing counterfactuals in a latent space of a deep generative model, enabling uncertainty-focused explanations and evaluation, with strong empirical and user-study support.

ABSTRACT

Both uncertainty estimation and interpretability are important factors for trustworthy machine learning systems. However, there is little work at the intersection of these two areas. We address this gap by proposing a novel method for interpreting uncertainty estimates from differentiable probabilistic models, like Bayesian Neural Networks (BNNs). Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold, such that a BNN becomes more confident about the input's prediction. We validate CLUE through 1) a novel framework for evaluating counterfactual explanations of uncertainty, 2) a series of ablation experiments, and 3) a user study. Our experiments show that CLUE outperforms baselines and enables practitioners to better understand which input patterns are responsible for predictive uncertainty.

Motivation & Objective

• Motivate the need for interpretable uncertainty estimates in probabilistic models, especially BNNs.
• Introduce CLUE to identify smallest latent-space changes that reduce model uncertainty while staying on the data manifold.
• Provide a computational evaluation framework to assess counterfactual explanations of uncertainty.
• Demonstrate CLUE’s effectiveness across tabular and image datasets and through human studies.

Proposed method

• Define an auxiliary latent-space deep generative model (DGM) and a VAE decoder to search for inputs that are close to the original yet yield lower uncertainty.
• Formulate CLUE objective L(z) = H(y|mu_theta(x|z)) + d(mu_theta(x|z), x0) and optimize over latent code z to obtain x_CLUE.
• Decode z_CLUE to x_CLUE using the DGM decoder, ensuring changes remain on-distribution and near the data manifold.
• Use an uncertainty estimator H that covers both epistemic and aleatoric components, applicable to regression and classification tasks.
• Employ an initialization strategy z0 = mu_phi(z|x0) and perform multiple runs to obtain diverse CLUEs, with early stopping based on convergence criteria.

Experimental results

Research questions

• RQ1 What is the smallest change to an input, staying on-distribution, that makes a BNN more certain about its prediction?
• RQ2 Can latent-space counterfactuals (CLUEs) provide meaningful, in-distribution explanations of high model uncertainty?
• RQ3 How do CLUEs compare to baseline uncertainty explanations and feature-importance methods in explaining uncertainty?
• RQ4 Does a ground-truth data-generating process allow objective evaluation of counterfactual explanations of uncertainty?

Key findings

• CLUE outperforms baselines in explaining uncertainty across multiple datasets and tasks.
• Using latent-space search ensures CLUEs remain in-distribution and interpretable, avoiding off-manifold perturbations.
• In computational evaluation, CLUE explains more ground-truth uncertainty than U-FIDO, while U-FIDO yields smaller input changes.
• CLUE achieves best performance in 8 out of 10 tasks in the evaluation metric based on log-likelihood under a ground-truth generative model.
• A user study shows CLUE improves practitioners’ ability to predict whether a BNN will be certain or uncertain for a new input, outperforming Local Sensitivity and other baselines.
• More expressive auxiliary DGMs yield CLUEs that are more relevant (smaller input changes for similar uncertainty reductions) and maintain information content about uncertainty.