[Paper Review] Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning

Qi Wu, Peng Wang+3 more|arXiv (Cornell University)|Nov 21, 2017

Multimodal Machine Learning Applications41 references16 citations

TL;DR

This paper proposes a novel adversarial learning framework that integrates generative adversarial networks (GANs) with a sequential co-attention mechanism to generate more human-like visual dialog responses. By training a discriminator to assess both the response and its associated attention-based reasoning, the generator produces more natural, contextually grounded answers, achieving state-of-the-art performance on the VisDial benchmark with a 2.14% improvement in recall@5 over prior methods.

ABSTRACT

The Visual Dialogue task requires an agent to engage in a conversation about an image with a human. It represents an extension of the Visual Question Answering task in that the agent needs to answer a question about an image, but it needs to do so in light of the previous dialogue that has taken place. The key challenge in Visual Dialogue is thus maintaining a consistent, and natural dialogue while continuing to answer questions correctly. We present a novel approach that combines Reinforcement Learning and Generative Adversarial Networks (GANs) to generate more human-like responses to questions. The GAN helps overcome the relative paucity of training data, and the tendency of the typical MLE-based approach to generate overly terse answers. Critically, the GAN is tightly integrated into the attention mechanism that generates human-interpretable reasons for each answer. This means that the discriminative model of the GAN has the task of assessing whether a candidate answer is generated by a human or not, given the provided reason. This is significant because it drives the generative model to produce high quality answers that are well supported by the associated reasoning. The method also generates the state-of-the-art results on the primary benchmark.

Motivation & Objective

• To address the limitations of existing Visual Dialogue models that generate overly terse, generic responses due to reliance on maximum likelihood estimation (MLE).
• To improve response quality by integrating adversarial learning with a discriminative model that evaluates both the response and its attention-based reasoning.
• To enable the generator to produce more natural, contextually consistent, and human-like dialogues by leveraging intermediate rewards and discriminative feedback.
• To evaluate the model's human-likeness through a Turing Test-style human study and benchmark against state-of-the-art methods on VisDial.

Proposed method

• A sequential co-attention mechanism is used to jointly attend to image features, question embeddings, and dialog history to generate contextually grounded representations.
• The generator model produces response sequences using an LSTM decoder conditioned on the co-attended multimodal features.
• A discriminator is trained to distinguish between human-generated and machine-generated dialogues, with access to the generator’s attention weights as a form of interpretability and reasoning evidence.
• Intermediate rewards are computed via Monte Carlo search to guide the generator using both global and step-by-step feedback from the discriminator.
• The final model incorporates teacher forcing after adversarial training to further stabilize and improve generation quality.
• The framework is jointly trained using reinforcement learning with the discriminator’s output serving as a reward signal to optimize the generator.

Experimental results

Research questions

• RQ1 Can adversarial learning improve the naturalness and coherence of visual dialogue responses beyond maximum likelihood estimation?
• RQ2 Does providing the discriminator access to the generator’s attention weights enhance the quality and interpretability of generated responses?
• RQ3 Can intermediate rewards during sequence generation lead to better response diversity and fluency compared to only global rewards?
• RQ4 To what extent can the generated responses pass a human evaluation as being indistinguishable from human-written dialogues?
• RQ5 How does the proposed method compare to state-of-the-art models in terms of standard benchmark metrics like recall@5 and accuracy?

Key findings

• The proposed CoAtt-GAN-w/ R_inte-TF model achieves a 2.14% improvement in recall@5 over the previous best-reported result on the VisDial v0.9 dataset.
• The model outperforms the strong MLE-based baseline CoAtt-G-MLE by 1.87% in recall@5, demonstrating the effectiveness of adversarial training.
• In a human evaluation, 49% of the responses generated by the proposed model passed the Turing Test, compared to 46% for the baseline and 39% for the Memory Network model.
• 45% of the responses were judged by human evaluators to be better than or equal to human-written responses, indicating high human-likeness.
• The discriminator correctly classified 70% of the generated responses as human-like, significantly above the 50% random chance baseline, showing the generator successfully fooled the discriminator.
• The ablation study confirms that intermediate rewards (R_inte) contribute the most to performance improvement, followed by teacher forcing, which further boosts results by 0.5% on average.