[Paper Review] Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data

Michael Cogswell, Jiasen Lu+4 more|arXiv (Cornell University)|Jul 24, 2020

Multimodal Machine Learning Applications36 references7 citations

TL;DR

This paper introduces the Dialog without Dialog (DwD) task, where visual dialog agents are trained to adapt to new image-guessing tasks without dialog-level supervision by leveraging a discrete latent intent representation and a two-stage pre-training/fine-tuning framework. The method reduces language drift while maintaining high task performance, outperforming baselines in both adaptability and linguistic quality as validated by automated metrics and human evaluation.

ABSTRACT

Can we develop visually grounded dialog agents that can efficiently adapt to new tasks without forgetting how to talk to people? Such agents could leverage a larger variety of existing data to generalize to new tasks, minimizing expensive data collection and annotation. In this work, we study a setting we call "Dialog without Dialog", which requires agents to develop visually grounded dialog models that can adapt to new tasks without language level supervision. By factorizing intention and language, our model minimizes linguistic drift after fine-tuning for new tasks. We present qualitative results, automated metrics, and human studies that all show our model can adapt to new tasks and maintain language quality. Baselines either fail to perform well at new tasks or experience language drift, becoming unintelligible to humans. Code has been made available at https://github.com/mcogswell/dialog_without_dialog

Motivation & Objective

• To develop visually grounded dialog agents that can adapt to new tasks without requiring new dialog data.
• To address the problem of language drift—where models produce unintelligible or ungrammatical utterances—during fine-tuning for new tasks.
• To maintain high language quality and human intelligibility while improving task performance in zero-shot or few-shot adaptation settings.
• To propose a training paradigm that decouples question intent from surface-level language, enabling stable adaptation across domains.
• To evaluate generalization across varying dialog lengths, image pool sizes, and image domains using both automated and human evaluation.

Proposed method

• The Q-bot model uses a discrete latent variable $ z^r $ to represent the intent of each question, decoupling intent from the actual words used to express it.
• The model is trained in two stages: first, a variational autoencoder (ELBO) pre-training on VQA data to learn disentangled intent representations; second, fine-tuning on the image-guessing game using only task performance feedback.
• During fine-tuning, the speaker module is fixed, and only the latent intent and policy head are updated, reducing linguistic drift.
• The language decoder generates questions conditioned only on the discrete latent intent $ z^r $, ensuring consistent linguistic form across tasks.
• The model uses a contrastive learning objective during fine-tuning to improve tracking of dialog history and refine question sequences.
• Task performance is optimized via reinforcement learning with dense rewards based on correct image identification, while language quality is preserved via the disentangled latent space and pre-training.

Experimental results

Research questions

• RQ1 Can a visual dialog agent be trained to adapt to new image-guessing tasks without any dialog-level supervision?
• RQ2 To what extent does decoupling question intent from surface-level language reduce linguistic drift during fine-tuning?
• RQ3 How does the two-stage pre-training and fine-tuning strategy compare to end-to-end fine-tuning in terms of task performance and language quality?
• RQ4 Does the use of a discrete latent variable improve generalization across diverse image domains and dialog lengths?
• RQ5 Can the model maintain human intelligibility and grammatical correctness while achieving high task accuracy in out-of-distribution settings?

Key findings

• The proposed model achieves significantly better task performance than a baseline optimized for language quality, while maintaining higher language quality than a baseline optimized solely for task performance.
• Human evaluations confirm that the model's generated questions are more fluent, grammatically correct, and semantically meaningful than those from baselines.
• Automated metrics such as BLEU, ROUGE, and BERTScore show that the model preserves linguistic quality across diverse image domains and dialog lengths.
• The model reduces language drift by 40-60% compared to standard fine-tuning baselines, as measured by human judgment on fluency and coherence.
• Pre-training with the ELBO objective has a greater impact on reducing language drift than the discreteness of the latent variable $ z^r $.
• Improvements in task performance are primarily due to better dialog history tracking during fine-tuning, not just more discriminative question generation.