[Paper Review] Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation

Vaishnavi Shrivastava, Percy Liang+1 more|arXiv (Cornell University)|Nov 15, 2023

Topic Modeling4 citations

TL;DR

This paper proposes a surrogate model approach to estimate confidence in state-of-the-art black-box LLMs like GPT-4 and Claude-v1.3, which do not expose internal probabilities. By using confidence scores from a weaker, open-source model (e.g., Llama 2) on the same input, the method achieves 84.6% average AUC on 12 question-answering datasets—surpassing linguistic confidence prompting and concurrent methods.

ABSTRACT

To maintain user trust, large language models (LLMs) should signal low confidence on examples where they are incorrect, instead of misleading the user. The standard approach of estimating confidence is to use the softmax probabilities of these models, but as of November 2023, state-of-the-art LLMs such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We first study eliciting confidence linguistically -- asking an LLM for its confidence in its answer -- which performs reasonably (80.5% AUC on GPT-4 averaged across 12 question-answering datasets -- 7% above a random baseline) but leaves room for improvement. We then explore using a surrogate confidence model -- using a model where we do have probabilities to evaluate the original model's confidence in a given question. Surprisingly, even though these probabilities come from a different and often weaker model, this method leads to higher AUC than linguistic confidences on 9 out of 12 datasets. Our best method composing linguistic confidences and surrogate model probabilities gives state-of-the-art confidence estimates on all 12 datasets (84.6% average AUC on GPT-4).

Motivation & Objective

• Address the challenge of estimating reliable confidence scores for state-of-the-art LLMs like GPT-4 and Claude-v1.3, which do not expose internal softmax probabilities.
• Improve upon linguistic confidence elicitation—prompting the model to self-report its confidence—by addressing its limitations of coarse-grained, repetitive outputs.
• Develop a practical, low-cost method to estimate confidence for black-box LLMs using accessible probabilities from smaller, open-source surrogate models.
• Demonstrate that surrogate model probabilities can effectively transfer and improve confidence estimation even when the surrogate model is weaker or different from the target model.
• Combine linguistic confidences and surrogate model probabilities to achieve state-of-the-art performance in selective classification across diverse benchmarks.

Proposed method

• Elicit linguistic confidence by prompting the target LLM (e.g., GPT-4) to rate its confidence in its own answer using various prompt formats.
• Use a surrogate model (e.g., Llama 2, text-davinci-003) that provides access to internal softmax probabilities to generate confidence scores for the same input.
• Compose confidence estimates by mixing linguistic confidence from the target model with probability-based confidence from the surrogate model.
• Apply a weighted combination of linguistic and surrogate confidence scores to improve separation between correct and incorrect predictions.
• Use selective classification AUC as the primary evaluation metric across 12 diverse question-answering datasets including MMLU, TruthfulQA, and MedQA.
• Validate robustness across multiple models (GPT-4, Claude-v1.3, GPT-3.5, Llama 2, text-davinci-003) and prompt variations, including chain-of-thought and few-shot formats.

Figure 1: Our goal is to provide good confidence estimates for state-of-the-art LLMs like GPT-4 and Claude-v1.3 which currently do not give access to their internal probabilities. One natural approach (GPT-4 Linguistic) is to prompt the model asking for its confidence. Interestingly, we find that ta

Experimental results

Research questions

• RQ1 Can linguistic confidence prompting reliably estimate model uncertainty when internal probabilities are unavailable?
• RQ2 To what extent can confidence estimates from a weaker, open-source surrogate model improve confidence estimation for stronger black-box LLMs like GPT-4?
• RQ3 Why does confidence transfer between models even when the surrogate model is significantly smaller or less accurate?
• RQ4 How do linguistic confidences and surrogate model probabilities complement each other in improving overall confidence estimation?
• RQ5 Can a simple mixture of linguistic and surrogate confidence scores outperform more complex, expensive methods like self-consistency?

Key findings

• Linguistic confidence prompting on GPT-4 achieves an average AUC of 80.5% across 12 datasets—7% above random baseline but significantly below model probabilities when available.
• Surrogate confidence estimation using Llama 2’s probabilities improves GPT-4’s average AUC to 82.1%, outperforming linguistic confidences alone.
• Combining linguistic confidences with surrogate model probabilities increases GPT-4’s average AUC to 83.4%, demonstrating complementarity between the two signals.
• The best-performing method—mixing linguistic and surrogate confidence—achieves a state-of-the-art average AUC of 84.6% on GPT-4 across all 12 datasets.
• Even weaker surrogate models like Llama 2-13B or text-davinci-003 yield comparable or better AUC than linguistic confidences on GPT-4 and Claude-v1.3, indicating strong transferability.
• The method outperforms concurrent work (Xiong et al., 2023) that uses self-consistency, which requires five GPT-4 calls per input and is more computationally expensive.

Figure 2: Linguistic Confidence Prompt Instruction for the best linguistic confidence prompt (see exact prompt in Appendix A.4 ).