[Paper Review] Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges

Chenhang Cui, Yiyang Zhou+5 more|arXiv (Cornell University)|Nov 6, 2023

COVID-19 diagnosis using AI13 citations

TL;DR

This paper introduces the Bingo benchmark to analyze bias and interference-driven hallucinations in GPT-4V(ision) and other vision-language models, evaluates mitigation attempts, and reports persistent challenges.

ABSTRACT

While GPT-4V(ision) impressively models both visual and textual information simultaneously, it's hallucination behavior has not been systematically assessed. To bridge this gap, we introduce a new benchmark, namely, the Bias and Interference Challenges in Visual Language Models (Bingo). This benchmark is designed to evaluate and shed light on the two common types of hallucinations in visual language models: bias and interference. Here, bias refers to the model's tendency to hallucinate certain types of responses, possibly due to imbalance in its training data. Interference pertains to scenarios where the judgment of GPT-4V(ision) can be disrupted due to how the text prompt is phrased or how the input image is presented. We identify a notable regional bias, whereby GPT-4V(ision) is better at interpreting Western images or images with English writing compared to images from other countries or containing text in other languages. Moreover, GPT-4V(ision) is vulnerable to leading questions and is often confused when interpreting multiple images together. Popular mitigation approaches, such as self-correction and chain-of-thought reasoning, are not effective in resolving these challenges. We also identified similar biases and interference vulnerabilities with LLaVA and Bard. Our results characterize the hallucination challenges in GPT-4V(ision) and state-of-the-art visual-language models, and highlight the need for new solutions. The Bingo benchmark is available at https://github.com/gzcch/Bingo.

Motivation & Objective

• Identify and characterize hallucination causes in GPT-4V(ision) focusing on bias and interference.
• Create a comprehensive benchmark (Bingo) with failure and success cases across regions, OCR, and factual content.
• Evaluate how prompt-based mitigations (self-correction, chain-of-thought) affect hallucinations.
• Compare GPT-4V(ision) with other vision-language models (LLaVA, Bard) on the Bingo tasks.
• Discuss implications for robustness and future directions in vision-language modeling.

Proposed method

• Construct Bingo with 190 failure and 131 success instances pairing images with one or two questions.
• Categorize bias into region bias, OCR bias, and factual bias; categorize interference into image-to-image and text-to-image interference.
• Evaluate GPT-4V(ision) using human annotations (correct/incorrect) and report accuracy across categories.
• Analyze biases and interferences in LLaVA-1.5 and Bard for comparative context.
• Test mitigation approaches including self-correction prompts and chain-of-thought prompts to assess impact on hallucinations.
• Provide qualitative examples and discuss limitations of current mitigation strategies.

Experimental results

Research questions

• RQ1 What are the main sources (bias vs. interference) of hallucinations in GPT-4V(ision) and other VLMs?
• RQ2 How do region, OCR, and factual biases affect vision-language understanding across languages and regions?
• RQ3 How do image-to-image and text-to-image interferences impact model judgments on visual prompts?
• RQ4 Do self-correction or chain-of-thought prompting mitigate hallucinations in vision-language models?
• RQ5 How do GPT-4V(ision), LLaVA-1.5, and Bard compare on the Bingo benchmark in terms of bias and interference?

Key findings

• GPT-4V(ision) shows regional bias, performing better on Western/world images than non-Western images.
• OCR bias is evident; English and French text within images are handled better than other languages due to OCR detector limitations.
• Factual bias exists where models rely on learned facts rather than image evidence, with counterfactual cases causing high error rates.
• Image-to-image interference severely degrades performance when similar images are grouped together.
• Text-to-image interference causes the model to align with user claims rather than image content (sycophancy).
• Self-correction reduces hallucinations by about 16.56–16.9% across categories, but many errors remain; chain-of-thought prompting provides limited or no consistent benefits.