[Paper Review] Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning

Qiming Bao, G. Gendron+6 more|arXiv (Cornell University)|Oct 13, 2023

Topic Modeling4 citations

TL;DR

This paper proposes three augmented logical reasoning datasets—ReClor-plus, LogiQA-plus, and LogiQAv2-plus—by perturbing task structures through option shuffling and replacing correct answers with 'none of the other options are correct.' Despite strong performance on original datasets, LLMs like GPT-3.5 and GPT-4 show significant performance drops on these variants, revealing poor robustness. The study demonstrates that instruction fine-tuning and logic-driven data augmentation significantly improve generalization and robustness, especially on large training sets (>10,000 samples) with high perturbation ratios.

ABSTRACT

Large language models (LLMs), such as LLaMA, Alpaca, Vicuna, GPT-3.5 and GPT-4, have advanced the performance of AI systems on various natural language processing tasks to human-like levels. However, their generalisation and robustness when performing logical reasoning has not been sufficiently assessed. To comprehensively evaluate this ability, we develop three new logical reasoning datasets named "ReClor-plus", "LogiQA-plus" and "LogiQAv2-plus" that extend standard logical reasoning datasets to evaluate the robustness of the LLM's reasoning. For each, we create three subsets: the first with randomly shuffled options, the second with the correct choices replaced by "none of the other options is correct", and the third with a combination of shuffling and substitution. Experiments on these datasets show that these simple augmentations greatly hinder the models' performance. Despite their high performance on the original publicly available datasets, we find that all models perform poorly on these newly constructed datasets. We also demonstrate that introducing task variations into the training set can markedly improve the model's performance on both the original and our developed datasets. Finally, we show that applying logic-driven data augmentation for fine-tuning and prompting can enhance generalisation in both discriminative and generative models, offering a path to improving their robustness for tasks involving logical reasoning. Source code and data are made publicly available at https://github.com/Strong-AI-Lab/Logical-and-abstract-reasoning.

Motivation & Objective

• To investigate the generalization and robustness of large language models (LLMs) on logical reasoning tasks under structural perturbations.
• To address the risk of memorization and overfitting in existing logical reasoning datasets by introducing task structure variations.
• To evaluate whether instruction fine-tuning, prompting, and data augmentation can improve LLMs’ resilience to distributional shifts in reasoning tasks.
• To examine the impact of training data size and perturbation ratio on model generalization, particularly for generative and discriminative LLMs.
• To determine whether model size (e.g., LLaMA-7B to LLaMA-65B) correlates with improved reasoning robustness under zero-shot evaluation.

Proposed method

• The authors create three new datasets—ReClor-plus, LogiQA-plus, and LogiQAv2-plus—by applying two perturbations: random shuffling of answer options and replacement of the correct answer with 'none of the other options are correct'.
• Each dataset includes three subsets: original format, shuffled options, and replaced correct answer, enabling systematic evaluation of model robustness.
• Experiments are conducted on both discriminative (e.g., LReasoner, MERIt) and generative (e.g., GPT-3.5, GPT-4, LLaMA, Alpaca, Vicuna) LLMs across zero-shot and fine-tuned settings.
• Logic-driven data augmentation is applied during instruction fine-tuning to improve reasoning generalization, with prompting used to further enhance performance.
• The study evaluates model performance across varying data perturbation ratios (5% to 50%) on training sets, focusing on generalization to out-of-distribution structures.
• Model size comparisons are performed using LLaMA models from 7B to 65B parameters under zero-shot evaluation to assess scalability effects.

Experimental results

Research questions

• RQ1 Do large language models exhibit robust generalization on logical reasoning tasks when faced with structural variations such as shuffled answer options or replaced correct answers?
• RQ2 Can instruction fine-tuning and logic-driven data augmentation improve the robustness and generalization of both discriminative and generative LLMs on out-of-distribution reasoning tasks?
• RQ3 Does the size of the training dataset influence the effectiveness of data perturbation in enhancing model generalization for logical reasoning?
• RQ4 Is there a significant correlation between model size (e.g., LLaMA-7B to LLaMA-65B) and performance on perturbed logical reasoning tasks under zero-shot evaluation?
• RQ5 To what extent does CoT prompting alone improve robustness in logical reasoning, compared to fine-tuning and data augmentation strategies?

Key findings

• GPT-3.5 and GPT-4 show strong performance on original logical reasoning datasets but suffer significant performance drops—down to 11.3% accuracy—on the perturbed 'Shuffle-RepAns' format, indicating reliance on memorization or superficial patterns.
• Instruction fine-tuning enables discriminative LLMs to achieve permutation invariance, meaning their predictions remain consistent regardless of answer option order, suggesting improved reasoning over memorization.
• For training sets exceeding 10,000 samples, higher data perturbation ratios (e.g., 50%) with both shuffled options and replaced correct answers significantly improve generalization in generative LLMs like GPT-3.5 and GPT-4.
• Logic-driven data augmentation combined with prompting enhances the performance of both discriminative and generative LLMs, particularly on the LogiQAv2-plus dataset, demonstrating improved robustness.
• No significant performance difference is observed across LLaMA models from 7B to 65B parameters under zero-shot evaluation on the Shuffle-RepAns format, indicating model size alone does not determine robustness.
• The performance improvement from data perturbation is not effective for training sets smaller than 10,000 samples, highlighting a threshold effect for effective data augmentation.