[Paper Review] Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

Hanmeng Liu, Ruoxi Ning+4 more|arXiv (Cornell University)|Apr 7, 2023

Topic ModelingComputer Science103 citations

TL;DR

The paper benchmarks ChatGPT and GPT-4 on multiple logical reasoning tasks (multi-choice reading comprehension and natural language inference) across popular and new datasets, introduces LogiEval, and analyzes performance gaps, especially on out-of-distribution data.

ABSTRACT

Harnessing logical reasoning ability is a comprehensive natural language understanding endeavor. With the release of Generative Pretrained Transformer 4 (GPT-4), highlighted as "advanced" at reasoning tasks, we are eager to learn the GPT-4 performance on various logical reasoning tasks. This report analyses multiple logical reasoning datasets, with popular benchmarks like LogiQA and ReClor, and newly-released datasets like AR-LSAT. We test the multi-choice reading comprehension and natural language inference tasks with benchmarks requiring logical reasoning. We further construct a logical reasoning out-of-distribution dataset to investigate the robustness of ChatGPT and GPT-4. We also make a performance comparison between ChatGPT and GPT-4. Experiment results show that ChatGPT performs significantly better than the RoBERTa fine-tuning method on most logical reasoning benchmarks. With early access to the GPT-4 API we are able to conduct intense experiments on the GPT-4 model. The results show GPT-4 yields even higher performance on most logical reasoning datasets. Among benchmarks, ChatGPT and GPT-4 do relatively well on well-known datasets like LogiQA and ReClor. However, the performance drops significantly when handling newly released and out-of-distribution datasets. Logical reasoning remains challenging for ChatGPT and GPT-4, especially on out-of-distribution and natural language inference datasets. We release the prompt-style logical reasoning datasets as a benchmark suite and name it LogiEval.

Motivation & Objective

• Assess ChatGPT and GPT-4 capabilities in logical reasoning across diverse datasets.
• Compare performance against RoBERTa fine-tuning baseline on multi-choice reading comprehension and NLI tasks.
• Investigate robustness to out-of-distribution data and the impact of prompt design and context on reasoning.
• Introduce and promote LogiEval as a prompt-style benchmark suite for evaluating large language models.

Proposed method

• Evaluate on two task types: multi-choice reading comprehension and natural language inference (NLI).
• Use RoBERTa-base as a fine-tuned baseline; compare with API/UI outputs from ChatGPT and GPT-4.
• Apply instruction-prompt schemes for ChatGPT and GPT-4, with labels-based prompts for NLI and explicit task formats for MC reading comprehension.
• In-context learning experiments with GPT-4 to assess improvements from contextual examples within and across conversation windows.
• Explore zero-shot chain-of-thought prompting to assess impact on reasoning performance.

Experimental results

Research questions

• RQ1 How do ChatGPT and GPT-4 perform on established logical reasoning benchmarks (LogiQA, ReClor) and newer datasets (AR-LSAT)?
• RQ2 Do these models exhibit robust logical reasoning on out-of-distribution data, and how do they compare to RoBERTa baselines?
• RQ3 What is the effect of prompt design, instructions, and in-context context on reasoning performance of ChatGPT and GPT-4?
• RQ4 Can chain-of-thought prompting or in-conversation context improve GPT-4’s logical reasoning in practice?

Key findings

• ChatGPT outperforms RoBERTa baselines on many well-known logical reasoning benchmarks but declines on out-of-distribution data (e.g., AR-LSAT, LogiQA 2.0 ood).
• GPT-4 generally achieves higher accuracy than ChatGPT on most datasets, but also shows substantial drops on out-of-distribution data and some NLI tasks.
• ChatGPT and GPT-4 show relatively strong performance on LogiQA and ReClor, but still struggle with out-of-distribution and certain NLI datasets requiring deeper reasoning.
• In-context learning helps GPT-4 increase accuracy within the same conversation window, with notable gains when more examples are seen (contextual windows).
• Zero-shot Chain-of-Thought prompting yields improved results on LogiQA 2.0 ood when applied to GPT-4, indicating potential for reasoning-enhanced prompting.
• LogiEval is released as a prompt-style benchmark suite to evaluate prompt-based large language models.