[Paper Review] Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification

John Dougrez-Lewis, Mahmud Elahi Akhter+4 more|arXiv (Cornell University)|Feb 16, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This paper introduces a novel logical reasoning framework to decompose claim and rumor verification into atomic reasoning steps, using two manually annotated datasets—Wikipedia claims and Twitter rumors—to evaluate GPT-3.5-Turbo and GPT-4. Key findings show that while models excel at deductive reasoning, they struggle significantly with abductive reasoning, especially in real-world rumors, and Chain-of-Thought prompting improves explanation quality over zero-shot approaches.

ABSTRACT

Although LLMs have shown great performance on Mathematics and Coding related reasoning tasks, the reasoning capabilities of LLMs regarding other forms of reasoning are still an open problem. Here, we examine the issue of reasoning from the perspective of claim verification. We propose a framework designed to break down any claim paired with evidence into atomic reasoning types that are necessary for verification. We use this framework to create RECV, the first claim verification benchmark, incorporating real-world claims, to assess the deductive and abductive reasoning capabilities of LLMs. The benchmark comprises of three datasets, covering reasoning problems of increasing complexity. We evaluate three state-of-the-art proprietary LLMs under multiple prompt settings. Our results show that while LLMs can address deductive reasoning problems, they consistently fail in cases of abductive reasoning. Moreover, we observe that enhancing LLMs with rationale generation is not always beneficial. Nonetheless, we find that generated rationales are semantically similar to those provided by humans, especially in deductive reasoning cases.

Motivation & Objective

• To develop a systematic framework for decomposing claim and rumor verification into atomic reasoning steps.
• To address the lack of structured evaluation for LLM reasoning in real-world, complex rumor verification tasks.
• To investigate whether LLMs like ChatGPT perform reliably in abductive reasoning, which is critical for viral misinformation.
• To compare the effectiveness of zero-shot, zero-shot Chain-of-Thought, and manual Chain-of-Thought prompting in improving reasoning quality.
• To provide empirical evidence on the limitations of LLMs in high-stakes verification tasks, moving beyond hype to measurable capabilities.

Proposed method

• Proposes a logical reasoning framework that breaks down claims and evidence into reasoning paths, modes (deductive/abductive), and processes.
• Creates two manually annotated datasets: one from Wikipedia (claims with closed-world assumptions) and one from the PHEME dataset (real-world rumors with open-world complexity).
• Stratifies reasoning tasks by mode—deductive (e.g., mathematical, temporal) versus abductive (e.g., causal inference, plausibility estimation).
• Evaluates GPT-3.5-Turbo and GPT-4 under three prompting paradigms: Zero-Shot (ZS), Zero-Shot Chain-of-Thought (ZS CoT), and Manual Chain-of-Thought (Manual CoT).
• Uses 3-class classification (True, False, Unverified) to assess model performance, with ground-truth labels from expert-annotated reasoning paths.
• Employs qualitative analysis of explanations to assess reasoning quality, especially for abductive cases where model justification is critical.

Figure 1: Resolution of a multi-hop compound reasoning type rumour using our proposed framework. We apply two sets of mutually exclusive labels: { Abductive , Deductive , Inductive } and { Causal , Mathematical }.

Experimental results

Research questions

• RQ1 Can a systematic logical reasoning framework be designed to decompose claim and rumor verification into atomic, interpretable reasoning steps?
• RQ2 How do LLMs like GPT-3.5-Turbo and GPT-4 perform in verifying real-world rumors compared to simple factual claims?
• RQ3 Does Chain-of-Thought prompting improve the quality and accuracy of reasoning in abductive verification tasks compared to zero-shot inference?
• RQ4 To what extent do LLMs rely on heuristics or spurious correlations in abductive reasoning, especially when evidence is ambiguous or incomplete?
• RQ5 Are the reasoning processes of LLMs in verification tasks reflective of human-like reasoning, or do they exhibit systematic flaws such as Clever Hans behavior?

Key findings

• GPT-4 achieves 97.4% accuracy on the Wikipedia-based dataset under Manual CoT, but only 52.6% on the PHEME-based rumor dataset, indicating a significant drop in performance on real-world complexity.
• On abductive reasoning tasks, GPT-4 achieves only 33.3% accuracy under Zero-Shot prompting, improving slightly to 44.4% with Zero-Shot Chain-of-Thought, but still underperforming on complex, multi-part rumors.
• Manual Chain-of-Thought produces higher-quality explanations than Zero-Shot or Zero-Shot Chain-of-Thought, particularly in abductive cases where reasoning chains are more coherent and logically grounded.
• GPT-3.5-Turbo shows a similar trend, with 76.7% accuracy on deductive tasks but only 33.3% on abductive tasks under Zero-Shot, indicating a fundamental limitation in handling non-deductive inference.
• The model frequently misclassifies unverified claims as true or false due to overreliance on plausible-sounding but unsupported inferences, especially in the absence of strong evidence.
• Even with Chain-of-Thought prompting, models fail to consistently generate correct reasoning paths for abductive claims, suggesting that current LLMs do not perform human-like abductive reasoning and instead rely on pattern-based heuristics.