[Paper Review] Semantic Consistency for Assuring Reliability of Large Language Models

Harsh Raj, Vipul Gupta+2 more|arXiv (Cornell University)|Aug 17, 2023

Topic Modeling4 citations

TL;DR

This paper introduces a novel semantic consistency metric for evaluating large language models (LLMs) in open-ended text generation, moving beyond lexical similarity to capture true semantic equivalence. It proposes the Ask-to-Choose (A2C) prompting strategy, which boosts accuracy by up to 47% and semantic consistency by up to 7-fold on TruthfulQA, demonstrating strong alignment with human judgments and outperforming traditional metrics.

ABSTRACT

Large Language Models (LLMs) exhibit remarkable fluency and competence across various natural language tasks. However, recent research has highlighted their sensitivity to variations in input prompts. To deploy LLMs in a safe and reliable manner, it is crucial for their outputs to be consistent when prompted with expressions that carry the same meaning or intent. While some existing work has explored how state-of-the-art LLMs address this issue, their evaluations have been confined to assessing lexical equality of single- or multi-word answers, overlooking the consistency of generative text sequences. For a more comprehensive understanding of the consistency of LLMs in open-ended text generation scenarios, we introduce a general measure of semantic consistency, and formulate multiple versions of this metric to evaluate the performance of various LLMs. Our proposal demonstrates significantly higher consistency and stronger correlation with human evaluations of output consistency than traditional metrics based on lexical consistency. Finally, we propose a novel prompting strategy, called Ask-to-Choose (A2C), to enhance semantic consistency. When evaluated for closed-book question answering based on answer variations from the TruthfulQA benchmark, A2C increases accuracy metrics for pretrained and finetuned LLMs by up to 47%, and semantic consistency metrics for instruction-tuned models by up to 7-fold.

Motivation & Objective

• To address the lack of robust evaluation for semantic consistency in LLM-generated text sequences beyond single-token outputs.
• To develop a generalizable metric that captures semantic equivalence across paraphrased prompts, improving reliability in real-world NLG applications.
• To investigate the disconnect between accuracy and consistency in LLMs, which are often assumed to be correlated.
• To design a prompting strategy that enhances both semantic consistency and accuracy without compromising factual correctness.
• To validate the proposed metric and method against human judgments and existing lexical-based metrics.

Proposed method

• Proposes a multi-stage LLM-based pipeline using in-context learning to generate paraphrased questions and extract short answers via few-shot prompting.
• Employs an auxiliary LLM to score pairwise semantic equivalence of generated answers using diverse semantic agreement functions (e.g., entailment, paraphrase, BLEURT).
• Formulates a general semantic consistency metric based on aggregated pairwise semantic equivalences, enabling evaluation across diverse LLM architectures and decoding strategies.
• Introduces the Ask-to-Choose (A2C) prompting strategy, which structures prompts to guide models toward consistent, accurate answers by framing choices explicitly.
• Uses multiple semantic equivalence functions—paraphrase, entailment, contradiction, and lexical metrics—within the consistency framework to assess robustness and alignment with human preferences.
• Employs a six-LLM-call inference pipeline per question, with answer generation, shortening, and semantic scoring stages, to compute final consistency scores.

Experimental results

Research questions

• RQ1 How can semantic consistency in LLM-generated text sequences be measured more effectively than with lexical similarity?
• RQ2 To what extent does semantic consistency correlate with human judgments of output similarity?
• RQ3 Can prompting strategies improve both accuracy and semantic consistency in LLMs without degrading factual correctness?
• RQ4 Is there a disconnect between accuracy and consistency in LLMs, and if so, can it be reconciled through architectural or prompting interventions?
• RQ5 How do different semantic agreement functions (e.g., entailment, paraphrase) compare in measuring consistency and aligning with human preferences?

Key findings

• Larger LLMs exhibit higher semantic consistency, though they also show an inverse scaling effect where accuracy decreases with size.
• Semantic consistency and accuracy are independent properties; models can be consistent but inaccurate, or accurate but inconsistent.
• The proposed semantic consistency metric shows significantly higher correlation with human judgments than traditional lexical metrics like BLEU or ROUGE.
• The Ask-to-Choose (A2C) prompting strategy increases accuracy by up to 47% and semantic consistency by up to 7-fold on instruction-tuned models in closed-book QA.
• After applying A2C, consistency scores for accurate answers remain high or improve, with the PP metric rising from 0.87 to 0.88 across models.
• The pairwise correlation between human judgments and semantic consistency metrics (especially entailment and paraphrase) remains strong, with Entropy showing a slight improvement in correlation after A2C.