[Paper Review] Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

Bairu Hou, Yujian Liu+4 more|arXiv (Cornell University)|Nov 15, 2023

Topic Modeling4 citations

TL;DR

This paper proposes Input Clarification Ensembling, a novel framework for decomposing uncertainty in large language models (LLMs) into data (aleatoric) and model (epistemic) uncertainty without retraining. By generating multiple clarifications of the input and ensembling the LLM's predictions across them, the method achieves symmetric uncertainty decomposition akin to Bayesian Neural Networks, demonstrating accurate and reliable uncertainty quantification across multiple benchmarks.

ABSTRACT

Uncertainty decomposition refers to the task of decomposing the total uncertainty of a predictive model into aleatoric (data) uncertainty, resulting from inherent randomness in the data-generating process, and epistemic (model) uncertainty, resulting from missing information in the model's training data. In large language models (LLMs) specifically, identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability, but remains an important open research question. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling, which can be applied to any pre-trained LLM. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions. We show that, when aleatoric uncertainty arises from ambiguity or under-specification in LLM inputs, this approach makes it possible to factor an (unclarified) LLM's predictions into separate aleatoric and epistemic terms, using a decomposition similar to the one employed by Bayesian neural networks. Empirical evaluations demonstrate that input clarification ensembling provides accurate and reliable uncertainty quantification on several language processing tasks. Code and data are available at https://github.com/UCSB-NLP-Chang/llm_uncertainty.

Motivation & Objective

• To address the lack of effective uncertainty decomposition methods for black-box, large-scale LLMs.
• To overcome the infeasibility of training multiple model variants (as in Bayesian Neural Networks) due to the high cost of fine-tuning or retraining LLMs.
• To develop a method that decomposes total uncertainty into data (aleatoric) and model (epistemic) components using only inference-time input manipulations.
• To enable practical uncertainty quantification for improving LLM reliability, interpretability, and trustworthiness in real-world applications.

Proposed method

• The method generates multiple input clarifications by prompting a separate clarification model to rephrase ambiguous inputs, reducing data uncertainty.
• Each clarification is fed into the fixed LLM, and the model's predictions are ensembled to estimate the output distribution.
• The framework leverages a symmetric decomposition structure to BNNs, where model uncertainty is estimated from prediction variance across clarifications.
• Data uncertainty is quantified as the variance in predictions after input clarification, while model uncertainty is derived from the remaining variance in the original input.
• The approach avoids model parameter updates, relying solely on input-level perturbations and inference calls to the pre-trained LLM.
• A monotonicity check is performed to validate that clarified inputs yield lower data uncertainty than original ambiguous inputs.

Figure 1: Overview of the proposed uncertainty quantification method.

Experimental results

Research questions

• RQ1 Can uncertainty in LLMs be effectively decomposed into data and model uncertainty without retraining the model?
• RQ2 Does input clarification ensembling produce uncertainty estimates that are consistent with the symmetric decomposition structure of Bayesian Neural Networks?
• RQ3 Can the proposed method reliably detect ambiguous inputs and improve answer recall through clarification-based ensembling?
• RQ4 How does the performance of the uncertainty decomposition vary across datasets with different ambiguity levels?
• RQ5 To what extent can the clarification model improve uncertainty quantification and downstream performance?

Key findings

• The proposed method achieves a significant drop in data uncertainty after input clarification, validating the monotonicity of the uncertainty decomposition.
• On the AmbigInst dataset, where ambiguities are explicit, the clarification model generates effective clarifications, leading to strong ambiguity detection performance.
• On the AmbigQA dataset, which features more subtle ambiguities, the method still improves uncertainty quantification, though performance is more limited due to the subtlety of the ambiguity.
• The recall of correct answers increases consistently with the number of clarifications, demonstrating the method’s effectiveness in improving LLM reliability.
• Combining the method with ground-truth disambiguation further improves performance, confirming the value of the clarification module.
• The framework outperforms vanilla prompting without clarification, showing that uncertainty-aware input ensembling enhances answer coverage and reliability.

Figure 2: Data uncertainty distribution on the AmbigQA (Min et al., 2020 ) dataset using the BNN method. We use kernel density estimation to smooth the frequency distribution histogram. BNN is achieved by ensembling different in-context examples.