[Paper Review] Large Language Models Can Be Easily Distracted by Irrelevant Context

Freda Shi, Xinyun Chen+6 more|arXiv (Cornell University)|Jan 31, 2023

Topic Modeling103 citations

TL;DR

The paper introduces GSM-IC, a distractibility benchmark for arithmetic reasoning, showing that irrelevant context dramatically harms prompting methods; self-consistency and instructed prompting can mitigate but not eliminate the issue.

ABSTRACT

Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all information in the input context is relevant for solving the task. In this work, we investigate the distractibility of large language models, i.e., how the model problem-solving accuracy can be influenced by irrelevant context. In particular, we introduce Grade-School Math with Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant information in the problem description. We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included. We also identify several approaches for mitigating this deficiency, such as decoding with self-consistency and adding to the prompt an instruction that tells the language model to ignore the irrelevant information.

Motivation & Objective

• Motivate evaluation of LLMs under realistic, distractible input contexts where not all information is relevant.
• Construct GSM-IC, a GSM8K-derived dataset with inserted irrelevant sentences to measure model sensitivity.
• Assess state-of-the-art prompting techniques on GSM-IC and quantify distractibility across models.
• Identify mitigation strategies that improve robustness to irrelevant context (e.g., self-consistency, distractor exemplars, ignore-context instructions).

Proposed method

• Create GSM-IC by adding an irrelevant sentence to base GSM8K problems without altering the correct solution.
• Evaluate prompting techniques (CoT, 0-CoT, LtM, Program) with and without self-consistency on GSM-IC using code-davinci-002 and text-davinci-003.
• Analyze prompt designs, including exemplars with distractors and instruction-based prompting to ignore irrelevant context.
• Measure micro, macro, and normalized accuracies to quantify distractibility and robustness.
• Perform breakdown analyses to identify factors of irrelevant context (topic overlap, role-name overlap, number ranges) and their impact.
• Extend evaluation to DROP with a football example to test robustness in longer contexts.

Figure 1: Illustration of the considered factors when creating the GSM-IC dataset. Best viewed in color.

Experimental results

Research questions

• RQ1 How does the inclusion of irrelevant contextual information affect the accuracy of current prompting techniques on arithmetic reasoning tasks?
• RQ2 Can prompting strategies (self-consistency, exemplars with distractors, and ignore-context instructions) mitigate the distractibility induced by irrelevant information?
• RQ3 What factors of irrelevant context most influence model performance, and how does model architecture or prompting style modulate this sensitivity?
• RQ4 Do robustness improvements on GSM-IC transfer to other datasets/tasks (e.g., DROP) and to different model families?

Key findings

• All investigated prompting techniques are sensitive to irrelevant information, with macro accuracy dropping significantly (less than 30% consistently solved).
• Self-consistency substantially boosts robustness; with 20 samples per problem, correct answers appear in 99.7% of samples for some prompts.
• Exemplar distractors and ignore-context instructions consistently improve robustness across prompts and models.
• LtM is generally the most robust against irrelevant context for micro accuracy, but macro gains vary by model and prompting setup.
• Breakdown analysis shows topic overlap and in-topic distractors most harm macro accuracy; numbers alone have less impact than lexical overlap with the original problem.
• Instructed prompting (e.g., telling models to ignore irrelevant information) yields notable gains, and instruction type matters (the explicit ignore-context instruction is crucial).
• On DROP, LtM and its instructed variants provide improvements, indicating broader relevance beyond GSM-IC.

Figure 2: Prompt formats for the investigated techniques on the right, which are constructed from building blocks on the left (best viewed in color). The [Problem with Irrelevant Context] is obtained by adding an irrelevant sentence ( italic and underlined ) to the original problem description and i