[Paper Review] Knowledge Conflicts for LLMs: A Survey

Rongwu Xu, Zehan Qi+5 more|arXiv (Cornell University)|Mar 13, 2024

Library Science and Information Systems4 citations

TL;DR

This survey systematically categorizes and analyzes knowledge conflicts in large language models (LLMs), identifying three types: context-memory, inter-context, and intra-memory conflicts. It examines their causes, behavioral impacts on model outputs, and evaluates existing solutions, offering a comprehensive framework to improve LLM robustness and trustworthiness in real-world applications with external and parametric knowledge sources.

ABSTRACT

This survey provides an in-depth analysis of knowledge conflicts for large language models (LLMs), highlighting the complex challenges they encounter when blending contextual and parametric knowledge. Our focus is on three categories of knowledge conflicts: context-memory, inter-context, and intra-memory conflict. These conflicts can significantly impact the trustworthiness and performance of LLMs, especially in real-world applications where noise and misinformation are common. By categorizing these conflicts, exploring the causes, examining the behaviors of LLMs under such conflicts, and reviewing available solutions, this survey aims to shed light on strategies for improving the robustness of LLMs, thereby serving as a valuable resource for advancing research in this evolving area.

Motivation & Objective

• To systematically categorize and analyze knowledge conflicts in LLMs arising from the interaction between contextual and parametric knowledge.
• To investigate the root causes of these conflicts, including noisy or misleading external sources and inconsistencies in pre-training data.
• To examine how LLMs behave under different conflict types, particularly in generating hallucinated or contradictory outputs.
• To review and evaluate existing solutions for mitigating knowledge conflicts, highlighting gaps in current approaches.
• To identify future research directions, including explainability, multilinguality, and multimodal conflict resolution in LLMs.

Proposed method

• The survey introduces a taxonomy of three conflict types: context-memory (between user input/retrieved context and model’s parametric knowledge), inter-context (among multiple external context sources), and intra-memory (within model’s own parametric knowledge due to training data inconsistencies).
• It analyzes the lifecycle of knowledge conflicts as a nexus connecting causal triggers (e.g., noisy data, ambiguous prompts) to model behaviors (e.g., hallucinations, inconsistent responses).
• The paper reviews existing solutions, including retrieval-augmented generation (RAG) techniques, confidence calibration, and attention head analysis (e.g., PH3 for conflict resolution via path patching without fine-tuning).
• It emphasizes the need for microscopic analysis of internal model mechanisms—such as attention heads and neuron activations—during conflict resolution to improve interpretability.
• The survey draws analogies to Freudian psychoanalysis, advocating for understanding conflict origins to address root causes rather than symptoms.
• It proposes future research directions, including multilingual conflict detection, cross-lingual knowledge alignment, and multimodal conflict resolution across text, vision, and audio inputs.

Experimental results

Research questions

• RQ1 How do context-memory conflicts arise when external contextual knowledge contradicts a model’s parametric knowledge?
• RQ2 What are the causes and behavioral manifestations of inter-context conflicts when multiple external sources provide conflicting information?
• RQ3 Why do intra-memory conflicts emerge within LLMs, and how do they affect response consistency across different phrasings of the same question?
• RQ4 How do knowledge conflicts contribute to hallucinations and reduced model trustworthiness in real-world applications?
• RQ5 What are the limitations of current solutions, and how can future research improve robustness through explainability, multilinguality, and multimodal integration?

Key findings

• Knowledge conflicts are a major source of hallucinations and inconsistent outputs in LLMs, particularly when external context contradicts parametric knowledge.
• Context-memory conflicts are prevalent in retrieval-augmented LLMs, where noisy or adversarial documents can mislead model responses.
• Inter-context conflicts arise when multiple retrieved documents provide contradictory information, increasing uncertainty in model reasoning.
• Intra-memory conflicts stem from training data inconsistencies, causing LLMs to produce divergent answers to semantically equivalent questions.
• Current solutions often rely on assumed priors or heuristic rules, lacking granularity and generalization across diverse conflict types.
• Emerging techniques like Pruning Head via Path Patching (PH3) show promise in resolving conflicts without parameter updates by analyzing attention head contributions.