[Paper Review] Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection

Xiaodan Liang, Lisa Lee+1 more|arXiv (Cornell University)|Mar 8, 2017

Multimodal Machine Learning Applications25 references52 citations

TL;DR

VRL introduces a deep reinforcement learning framework with a variation-structured action graph to sequentially detect visual relationships and attributes, leveraging language priors and global context to scale to thousands of relation/attribute types. It achieves state-of-the-art results on VRD and Visual Genome, including zero-shot scenarios.

ABSTRACT

Despite progress in visual perception tasks such as image classification and detection, computers still struggle to understand the interdependency of objects in the scene as a whole, e.g., relations between objects or their attributes. Existing methods often ignore global context cues capturing the interactions among different object instances, and can only recognize a handful of types by exhaustively training individual detectors for all possible relationships. To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image. First, a directed semantic action graph is built using language priors to provide a rich and compact representation of semantic correlations between object categories, predicates, and attributes. Next, we use a variation-structured traversal over the action graph to construct a small, adaptive action set for each step based on the current state and historical actions. In particular, an ambiguity-aware object mining scheme is used to resolve semantic ambiguity among object categories that the object detector fails to distinguish. We then make sequential predictions using a deep RL framework, incorporating global context cues and semantic embeddings of previously extracted phrases in the state vector. Our experiments on the Visual Relationship Detection (VRD) dataset and the large-scale Visual Genome dataset validate the superiority of VRL, which can achieve significantly better detection results on datasets involving thousands of relationship and attribute types. We also demonstrate that VRL is able to predict unseen types embedded in our action graph by learning correlations on shared graph nodes.

Motivation & Objective

• Motivate holistic scene understanding by jointly detecting object relationships and attributes beyond individual detectors.
• Capture global interdependencies among objects using a directed semantic action graph built from language priors.
• Develop a variation-structured traversal to create small, adaptive action sets and enable efficient RL in a large action space.
• Resolve semantic ambiguity among object categories via an ambiguity-aware object mining scheme.
• Incorporate global image context and history embeddings to improve sequential reasoning in RL.

Proposed method

• Construct a directed semantic action graph where nodes represent object categories, attributes, and predicates, connected by edges encoding semantic correlations.
• Use a variation-structured traversal to dynamically form small action sets for each step, reducing the effective action space from thousands to a manageable subset.
• Apply a three-action decision process per step: select subject attribute, select predicate, and select next object category with ambiguity-aware mining.
• Encode a state vector that combines object-level features, whole-image features, and a history phrase embedding from a Skip-thought model.
• Train three separate deep Q-networks (one for attributes, one for predicates, one for object categories) with shared replay memory and target networks, using an epsilon-greedy learning strategy.
• Evaluate on VRD and Visual Genome with recall-based metrics, including zero-shot evaluations; compare against state-of-the-art methods that use shared vs. individual detectors.

Experimental results

Research questions

• RQ1 Can a Deep VRL framework effectively discover and localize thousands of potential relationships and attributes in complex scenes?
• RQ2 Does a variation-structured action graph improve learning efficiency and generalization over large, combinatorial action spaces?
• RQ3 How do language priors and global context cues impact relation and attribute detection performance?
• RQ4 Can the model generalize to unseen relationship and attribute types (zero-shot) within the action graph framework?

Key findings

• VRL achieves higher recall@100 and recall@50 on relationship and attribute detection in VRD and Visual Genome compared to strong baselines, including zero-shot scenarios.
• The variation-structured traversal dramatically reduces the effective action space (e.g., predicate actions reduced from 347 to about 15 on average) and improves learning stability and speed.
• Ambiguity-aware object mining improves category selection by leveraging scene context, leading to more concrete predictions (e.g., distinguishing man from skier, hat from helmet).
• History phrase embeddings and global image context significantly boost performance compared to variants without history or with simplistic history representations.
• VRL demonstrates strong zero-shot capabilities by predicting unseen types embedded in the action graph, leveraging shared graph nodes and learned correlations.
• On VRD, VRL outperforms state-of-the-art by substantial margins for both relationship phrase detection and relationship detection.