[Paper Review] LM vs LM: Detecting Factual Errors via Cross Examination

Roi Cohen, May Hamri+2 more|arXiv (Cornell University)|May 22, 2023

Topic Modeling4 citations

TL;DR

This paper proposes LMvLM, a zero-shot factuality detection method that uses one language model (the Examiner) to cross-examine another (the Examinee) by posing multi-turn questions to uncover inconsistencies in its claims. The method achieves over 70% detection of factual errors with >80% precision across multiple LMs and benchmarks, outperforming strong baselines by a large margin.

ABSTRACT

A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically. Inspired by truth-seeking mechanisms in law, we propose a factuality evaluation framework for LMs that is based on cross-examination. Our key idea is that an incorrect claim is likely to result in inconsistency with other claims that the model generates. To discover such inconsistencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which introduces questions to discover inconsistencies. We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that it outperforms existing methods and baselines, often by a large gap. Our results demonstrate the potential of using interacting LMs for capturing factual errors.

Motivation & Objective

• To address the critical problem of factual hallucinations in modern language models, which undermine reliability and usability.
• To develop a zero-shot, interactive method for detecting factual errors without relying on external knowledge or fine-tuning.
• To explore whether self-consistency checks via multi-turn interaction between LMs can effectively reveal factual inaccuracies.
• To evaluate the effectiveness of this approach across diverse LMs and factual question-answering benchmarks.
• To analyze the strategies employed by the Examiner LM during cross-examination to detect inconsistencies.

Proposed method

• The method uses two language models: one (Examinee) generates a claim, and another (Examiner) conducts a multi-turn cross-examination to detect inconsistencies.
• The Examiner is prompted to generate questions that probe the veracity of the claim, including paraphrasing, validation of implications, decomposition, and requests for evidence.
• The interaction proceeds in three stages: setup, questioning, and conclusion, with context from prior turns included in each prompt.
• The Examiner evaluates the Examinee’s responses and issues a final verdict on whether the original claim is factually correct or not.
• The approach is implemented in a zero-shot setting using prompt engineering to assign distinct roles to the same or different LMs.
• The method relies on the assumption that factually incorrect claims will lead to inconsistent responses under scrutiny, revealing internal contradictions.

Figure 1: An example of our LMvLM approach. The first line shows the statement made by the Examinee LLM. Then an interaction between the Examiner and Examinee takes place, and the Examiner arrives at a conclusion whether the original statement was correct or not (here it concludes that it was a fals

Experimental results

Research questions

• RQ1 Can a language model effectively detect factual errors in another language model’s claims through multi-turn questioning?
• RQ2 What types of questioning strategies do Examiner LMs employ to uncover inconsistencies in Examinee responses?
• RQ3 How does the performance of LMvLM compare to existing factuality detection baselines across multiple LMs and benchmarks?
• RQ4 To what extent do logical inconsistencies, such as contradictory answers to rephrased questions, signal factual errors?
• RQ5 Why does the method perform better with certain LMs (e.g., ChatGPT) than others (e.g., GPT-3), and what role does reasoning capability play?

Key findings

• LMvLM detects over 70% of factual errors across multiple benchmarks while maintaining a precision of over 80%, significantly outperforming strong baselines.
• The method achieves high detection performance even in a zero-shot setting, without fine-tuning or external knowledge.
• Examiner LMs frequently use strategies such as logical decomposition (75% of cases with ChatGPT), validation of implications (87.5% of correct detections with ChatGPT), and question paraphrasing to expose inconsistencies.
• Inconsistencies such as contradictory answers to rephrased questions occur in 14% of cases where the Examinee is correct but in over 50% of cases where the claim is false and detected.
• The failure cases are often due to the Examinee generating a coherent set of false facts that are mutually consistent, making inconsistencies hard to detect.
• ChatGPT outperforms GPT-3 in part due to its greater use of logical decomposition and validation strategies during cross-examination.

Figure 2: The three-stage process of cross-examination between the Examiner and Examinee , where the factuality of a claim $C$ generated by Examinee is estimated by Examiner .