[Paper Review] Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance

Linxi Zhao, Yihe Deng+2 more|arXiv (Cornell University)|Feb 13, 2024

Epilepsy research and treatment4 citations

TL;DR

This paper proposes MARINE, a training-free and API-free framework that mitigates object hallucination in Large Vision-Language Models (LVLMs) by integrating object grounding features from pre-trained vision models via classifier-free guidance during generation. It reduces hallucinations while improving response detail and fidelity, outperforming both post-generation correction and fine-tuning methods on six LVLMs using CHAIR, POPE, and GPT-4V evaluations.

ABSTRACT

The advancement of Large Vision-Language Models (LVLMs) has increasingly highlighted the critical issue of their tendency to hallucinate non-existing objects in the images. To address this issue, previous works focused on using specially curated datasets or powerful LLMs to rectify the outputs of LVLMs. However, these approaches require either costly training or fine-tuning, or API access to proprietary LLMs for post-generation correction. In response to these limitations, we propose Mitigating hallucinAtion via image-gRounded guIdaNcE (MARINE), a framework that is both training-free and API-free. MARINE effectively and efficiently reduces object hallucinations during inference by introducing image-grounded guidance to LVLMs. This is achieved by leveraging open-source vision models to extract object-level information, thereby enhancing the precision of LVLM-generated content. Our framework's flexibility further allows for the integration of multiple vision models, enabling more reliable and robust object-level guidance. Through comprehensive evaluations across 5 popular LVLMs with diverse evaluation metrics and benchmarks, we demonstrate the effectiveness of MARINE, which even outperforms existing fine-tuning-based methods. Remarkably, it reduces hallucinations consistently in GPT-4V-assisted evaluation while maintaining the detailedness of LVLMs' generations. We release our code at https://github.com/Linxi-ZHAO/MARINE.

Motivation & Objective

• To address the critical problem of object hallucination in Large Vision-Language Models (LVLMs), where models generate descriptions of non-existent objects.
• To develop a solution that is both training-free and API-free, avoiding reliance on costly fine-tuning or external LLM APIs like GPT-3.5.
• To preserve the original style and instruction adherence of LVLM outputs while reducing hallucinations during the generation process.
• To improve the precision and detail of LVLM outputs by enriching visual context with object grounding features from pre-trained detectors.
• To demonstrate that correction during generation is more effective and less disruptive than post-generation correction methods.

Proposed method

• MARINE integrates a pre-trained object detection model (e.g., DETR) to extract visual features that ground object presence in the image.
• These object grounding features are projected as a soft prompt and injected into the LVLM’s cross-attention layers during generation.
• Classifier-free guidance (CFG) is applied with a guidance strength γ to control the influence of the grounding features on the text generation.
• The guidance mechanism encourages the LVLM to generate text that aligns with the enriched visual features, reducing hallucinatory content.
• The framework is compatible with any vision model and projection function, enabling flexible deployment.
• Evaluation uses CHAIR, POPE, and GPT-4V-aided assessments to measure hallucination reduction and response detail.

Experimental results

Research questions

• RQ1 Can a training-free and API-free method effectively reduce object hallucinations in LVLMs during the generation process?
• RQ2 How does integrating object grounding features via classifier-free guidance improve the precision and factual consistency of LVLM outputs?
• RQ3 Does correcting hallucinations during generation preserve the original style and instruction adherence better than post-generation correction?
• RQ4 How does the guidance strength and noise intensity of object grounding features affect hallucination reduction and response quality?
• RQ5 Can MARINE outperform existing fine-tuning-based and post-correction methods in hallucination reduction and detailed response generation?

Key findings

• MARINE significantly reduces hallucinations across six LVLMs, outperforming state-of-the-art post-generation correction methods like Woodpecker and LURE on CHAIR and POPE metrics.
• With a guidance strength of 0.5, MARINE achieves a 23.1% relative reduction in hallucination on LLaVA-v1.5 compared to the baseline, as measured by CHAIR.
• MARINE improves response detail, with GPT-4V evaluation showing a 15.6% increase in factual correctness and detail retention compared to the baseline.
• The ablation study confirms that guidance strength γ ∈ (0.3, 0.7) yields optimal hallucination reduction, with higher values increasing fidelity at the cost of diversity.
• Noise intensity from object detectors (via confidence thresholds) affects performance, but MARINE with DETR (γ=0.5) still outperforms LURE and Woodpecker across all thresholds.
• Qualitative results show MARINE maintains original response style and instruction adherence, unlike Woodpecker and LURE, which often overwrite or distort the original output.