[Paper Review] Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis

Li Du, Yequan Wang+5 more|arXiv (Cornell University)|Sep 11, 2023

Topic Modeling4 citations

TL;DR

This paper proposes an association analysis framework to jointly quantify hallucination rates in large language models (LLMs) and attribute them to underlying risk factors, controlling for confounders. It identifies commonsense memorization, relational reasoning complexity, and instruction-following conflicts as key contributors to hallucination, with GPT-4 showing high hallucination rates due to strong language modeling conflicting with human instructions.

ABSTRACT

Although demonstrating superb performance on various NLP tasks, large language models (LLMs) still suffer from the hallucination problem, which threatens the reliability of LLMs. To measure the level of hallucination of LLMs, previous works first categorize the hallucination according to the phenomenon similarity, then quantify the proportion that model outputs contain hallucinatory contents. However, such hallucination rates could easily be distorted by confounders. Moreover, such hallucination rates could not reflect the reasons for the hallucination, as similar hallucinatory phenomena may originate from different sources. To address these issues, we propose to combine the hallucination level quantification and hallucination reason investigation through an association analysis, which builds the relationship between the hallucination rate of LLMs with a set of risk factors. In this way, we are able to observe the hallucination level under each value of each risk factor, examining the contribution and statistical significance of each risk factor, meanwhile excluding the confounding effect of other factors. Additionally, by recognizing the risk factors according to a taxonomy of model capability, we reveal a set of potential deficiencies in commonsense memorization, relational reasoning, and instruction following, which may further provide guidance for the pretraining and supervised fine-tuning process of LLMs to mitigate the hallucination.

Motivation & Objective

• To address the limitations of existing hallucination quantification methods that are distorted by confounders and fail to explain causes.
• To simultaneously measure hallucination levels and attribute them to specific risk factors using statistical association modeling.
• To identify fundamental model capability deficiencies—such as in commonsense memorization, relational reasoning, and instruction following—that contribute to hallucination.
• To guide pretraining and fine-tuning processes by revealing which model capabilities are most vulnerable to hallucination.

Proposed method

• Model the hallucination probability as a function of multiple risk factors using regression-based association analysis.
• Control for confounding effects by comparing hallucination rates across fixed levels of each risk factor.
• Define risk factors based on a taxonomy of model capabilities, including entity frequency, relational complexity, and instruction-language modeling conflict.
• Use log-likelihood differences between correct and hallucinated outputs as a proxy for instruction-following fidelity.
• Apply the method across diverse tasks (e.g., c-NLI, Ruletaker) to assess hallucination across different reasoning and knowledge types.
• Statistically evaluate the significance and contribution of each risk factor using regression coefficients and p-values.

Experimental results

Research questions

• RQ1 How does the hallucination rate of LLMs vary across different levels of risk factors such as entity frequency and relational complexity?
• RQ2 To what extent do conflicts between language modeling objectives and human instructions contribute to hallucination?
• RQ3 Which model capabilities—commonsense memorization, relational reasoning, or instruction following—are most strongly linked to hallucination?
• RQ4 Can hallucination be quantified and attributed in a way that controls for confounding variables in dataset composition?

Key findings

• LLMs exhibit higher hallucination rates on entities and relations with lower frequency in training data, indicating poor memorization of rare knowledge.
• For relational reasoning tasks, increasing the number of theories, facts, or arguments significantly raises hallucination rates—e.g., a one-unit increase in arguments raises hallucination by 1.13× (exp(0.12)).
• GPT-4 shows a U-shaped hallucination pattern: high rates despite being state-of-the-art, due to strong language modeling conflicting with human instructions.
• The likelihood decrease upon counterfactual statements is significantly correlated with hallucination rates, suggesting that strong language modeling can undermine instruction following.
• Even GPT-4 achieves only 57% accuracy on correct reasoning and answer generation in complex relational tasks (Ruletaker), indicating persistent challenges in relational reasoning.
• The conflict between language modeling objectives and human instructions is a statistically significant risk factor, with a coefficient of 0.11 for GPT-4.