[Paper Review] Woodpecker: Hallucination Correction for Multimodal Large Language Models

Shukang Yin, Chaoyou Fu+8 more|arXiv (Cornell University)|Oct 24, 2023

Topic Modeling44 references22 citations

TL;DR

Woodpecker is a training-free framework that post-processes MLLM outputs to correct image-grounded hallucinations, using a five-stage pipeline with bounding-box evidence for interpretability.

ABSTRACT

Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content. In order to mitigate hallucinations, existing studies mainly resort to an instruction-tuning manner that requires retraining the models with specific data. In this paper, we pave a different way, introducing a training-free method named Woodpecker. Like a woodpecker heals trees, it picks out and corrects hallucinations from the generated text. Concretely, Woodpecker consists of five stages: key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction. Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages. We evaluate Woodpecker both quantitatively and qualitatively and show the huge potential of this new paradigm. On the POPE benchmark, our method obtains a 30.66%/24.33% improvement in accuracy over the baseline MiniGPT-4/mPLUG-Owl. The source code is released at https://github.com/BradyFU/Woodpecker.

Motivation & Objective

• Motivate the need to mitigate visual hallucinations in multimodal large language models (MLLMs) without retraining.
• Propose a training-free, plug-in correction pipeline that diagnoses and amends hallucinations while providing grounding evidence.
• Show interpretability by exposing intermediate outputs and bounding-box evidence for verification.
• Evaluate the framework on established benchmarks (POPE, MME, LLaVA-QA90) and compare gains against baseline MLLMs.

Proposed method

• Five-stage correction pipeline: (1) key concept extraction to identify main objects mentioned in the generated text; (2) question formulation around objects and attributes; (3) visual knowledge validation using vision models to answer QA prompts; (4) visual claim generation to build a structured visual knowledge base with object-level and attribute-level claims; (5) hallucination correction where an LLM revises the response guided by the visual knowledge base and attaches bounding boxes as evidence.
• Grounding-based components: open-set object detector (Grounding DINO) for object existence/count, and VQA model (BLIP-2-FlanT5 XXL) for attribute questions.
• LLM-driven: GPT-3.5-turbo used for key concept extraction, question formulation, and final correction; prompts are designed for interpretability and to preserve bounding-box evidence.
• Training-free design: relies on off-the-shelf models (no model retraining of the MLLM) to enable easy integration with different MLLMs.
• Evidence augmentation: includes bounding boxes in corrected outputs to facilitate visual fact-checking.

Experimental results

Research questions

• RQ1 Can a training-free, plug-in framework reduce object-level and attribute-level hallucinations in MLLMs without retraining?
• RQ2 How does Woodpecker perform across different MLLMs and datasets (POPE, MME, LLaVA-QA90) in terms of accuracy and detailedness?
• RQ3 To what extent does grounding evidence (bounding boxes) improve interpretability and verification of corrected results?

Key findings

• On POPE, Woodpecker improves accuracy for baseline MiniGPT-4 and mPLUG-Owl by 30.66% and 24.33%, respectively.
• On POPE, Woodpecker boosts accuracy from 54.67%/62% to 85.33%/86.33% for MiniGPT-4/mPLUG-Owl respectively.
• On MME, object-level corrections yield large score gains, ranging from +65 over LLaVA to +101.66 over MiniGPT-4, and attribute-level gains (e.g., color) significantly improve after correction.
• On LLaVA-QA90 with GPT-4V-aided evaluation, corrected outputs achieve higher accuracy and detailedness, e.g., accuracy/detailedness gains across models (examples: LLaVA from 7.1 to 7.8 in accuracy and 7.1 to 8.6 in detailedness under corrected condition).
• The correction accuracy of the framework is 79.2% across the tested MME scenarios, with relatively low omission and mis-correction rates.
• The bounding-box evidence enhances the detail in responses, contributing to improved detailedness in GPT-4V-aided evaluation.