[Paper Review] Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment

Pritam Sarkar, Sayna Ebrahimi+4 more|arXiv (Cornell University)|May 28, 2024

Functional Brain Connectivity StudiesNeuroscience3 citations

TL;DR

This paper proposes HALVA, a data-augmented contrastive tuning method to mitigate object hallucination in Multimodal Large Language Models (MLLMs) by generating hallucinated responses via data augmentation and applying a contrastive loss at the token level to reduce the likelihood of incorrect tokens. The approach effectively reduces hallucinations without degrading general vision-language performance, requiring minimal fine-tuning and no inference overhead.

ABSTRACT

Despite their significant advancements, Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination. In this work, we address object hallucinations in MLLMs, where information is generated about an object not present in the input image. We introduce Data-augmented Phrase-level Alignment (DPA), a novel loss which can be applied to instruction-tuned off-the-shelf MLLMs to mitigate hallucinations, while preserving their general vision-language capabilities. To fine-tune MLLMs with DPA, we first generate a set of `hallucinated' and `correct' response pairs through generative data augmentation by selectively altering the ground-truth information of the correct responses at a phrase level. The DPA loss is then used to train MLLMs to reduce the likelihood of hallucinated phrases compared to the correct ones. Our thorough evaluation on various benchmarks confirms the effectiveness of DPA in mitigating hallucination while retaining the out-of-the-box performance of the MLLMs on general tasks. For instance, MLLMs finetuned with DPA, which we refer to as Hallucination Attenuated Language and Vision Assistant (HALVA), improve F1 by up to 13.4% on hallucination visual question-answering and reduce the hallucination rate by up to 4.2% on image description tasks.

Motivation & Objective

• To address object hallucination in MLLMs, where models generate descriptions of non-existent objects or attributes.
• To develop a method that mitigates hallucinations without increasing inference time or requiring retraining from scratch.
• To preserve or improve general vision-language performance after hallucination mitigation.
• To enable application to off-the-shelf MLLMs without extensive data or architectural changes.
• To reduce reliance on costly inference-time corrections or large-scale pretraining.

Proposed method

• Generative data augmentation is used to create hallucinated responses by selectively altering factual attributes in ground-truth captions.
• A contrastive loss is applied at the token level between factual and hallucinated tokens to increase the relative likelihood of correct tokens.
• The method employs a KL-divergence constraint with a reference model (the base MLLM) to preserve general capabilities during tuning.
• The framework is applied as a lightweight fine-tuning procedure on pretrained MLLMs, enabling direct deployment on off-the-shelf models.
• The approach is evaluated on both generative and discriminative vision-language tasks to assess hallucination reduction and generalization.
• The training process is fast and efficient, requiring minimal compute and no additional inference infrastructure.

Experimental results

Research questions

• RQ1 Can a contrastive tuning method reduce object hallucinations in MLLMs without degrading their general vision-language capabilities?
• RQ2 How effective is data-augmented hallucinated response generation in improving hallucination detection at inference time?
• RQ3 Does the proposed method maintain or improve performance on standard vision-language benchmarks after fine-tuning?
• RQ4 Can the method be applied to off-the-shelf MLLMs with minimal computational cost and no inference overhead?
• RQ5 To what extent does the method mitigate hallucinations caused by object co-occurrence patterns in training data?

Key findings

• HALVA significantly reduces object hallucinations on both generative and discriminative vision-language benchmarks, outperforming baseline methods in hallucination mitigation.
• The method maintains or improves performance on standard vision-language benchmarks such as VQAv2, GQA, and OK-ViCa, indicating no degradation in general capabilities.
• HALVA effectively reduces the tendency of MLLMs to answer 'Yes' to all questions due to positive instruction bias, as shown in qualitative comparisons.
• The model successfully mitigates hallucinations caused by co-occurrence patterns, such as falsely describing 'pitcher' with 'batter' and 'umpire' in a baseball scene.
• The contrastive tuning process is efficient, requiring minimal training time and no additional inference cost, making it suitable for production deployment.
• Qualitative results show HALVA accurately describes object positions and attributes, while the base model LLaVA-v1.5 frequently hallucinates objects like 'carrot', 'people', or 'handbag' in scenes where they are absent.