[Paper Review] A Joint Speaker-Listener-Reinforcer Model for Referring Expressions

Licheng Yu, Hao Tan+2 more|arXiv (Cornell University)|Dec 30, 2016

Multimodal Machine Learning Applications11 references5 citations

TL;DR

This paper proposes a joint speaker-listener-reinforcer model for referring expression generation and comprehension, where the speaker generates expressions, the listener understands them, and a reinforcement module optimizes for discriminative, unambiguous expressions. The framework achieves state-of-the-art performance on three MSCOCO-based referring expression datasets through end-to-end training and a reward-based reranking mechanism.

ABSTRACT

Referring expressions are natural language constructions used to identify particular objects within a scene. In this paper, we propose a unified framework for the tasks of referring expression comprehension and generation. Our model is composed of three modules: speaker, listener, and reinforcer. The speaker generates referring expressions, the listener comprehends referring expressions, and the reinforcer introduces a reward function to guide sampling of more discriminative expressions. The listener-speaker modules are trained jointly in an end-to-end learning framework, allowing the modules to be aware of one another during learning while also benefiting from the discriminative reinforcer's feedback. We demonstrate that this unified framework and training achieves state-of-the-art results for both comprehension and generation on three referring expression datasets. Project and demo page: https://vision.cs.unc.edu/refer

Motivation & Objective

• To develop a unified framework that jointly models speaker and listener behaviors in referring expression tasks.
• To address ambiguity in referring expressions by introducing a discriminative reward-based reinforcer module.
• To improve both referring expression generation and comprehension through end-to-end training with mutual supervision.
• To demonstrate robust performance under realistic conditions using object detectors instead of ground-truth bounding boxes.

Proposed method

• The model integrates three components: a speaker (LSTM-based decoder) generating referring expressions, a listener (embedding-based model) comprehending expressions, and a reinforcer using reinforcement learning to optimize for discriminative expressions.
• The speaker and listener are trained jointly in an end-to-end manner, enabling mutual feedback during learning.
• The reinforcer module employs a discriminative reward function that penalizes ambiguous expressions, guiding the speaker toward more unique descriptions.
• A reranking mechanism is applied using the listener and reinforcer to globally optimize expressions across all objects in a scene.
• The framework uses a joint-embedding space for comprehension and a conditional language model for generation, with Bayesian inference used to compute posterior probabilities for localization.
• The model is trained using a combination of cross-entropy loss for generation and reinforcement learning with a reward signal based on discriminative accuracy.

Experimental results

Research questions

• RQ1 Can a unified framework jointly optimize referring expression generation and comprehension with mutual supervision?
• RQ2 How does incorporating a discriminative reward function improve the quality and uniqueness of generated expressions?
• RQ3 To what extent does end-to-end training of speaker and listener modules improve performance compared to isolated training?
• RQ4 Does the joint model maintain strong performance when using automatic object detection instead of ground-truth bounding boxes?

Key findings

• The full model with reranking achieves 76.95% and 78.10% accuracy on RefCOCO Test A and B, respectively, outperforming prior state-of-the-art methods.
• On RefCOCO+, the model reaches 58.85% and 58.20% on Test A and B, setting new SOTA results.
• The addition of the reinforcer module improves comprehension performance by up to 2.8% over models without it, especially in ambiguous scenarios.
• Human evaluation confirms that the joint model with reranking generates more accurate and unambiguous expressions than baseline models.
• The model maintains strong performance even when using SSD object detection (0.3 confidence threshold), showing robustness to detection errors.
• The ablation study confirms that each component—listener, reinforcer, and reranking—contributes independently to performance gains.