[Paper Review] Learning where to look: Semantic-Guided Multi-Attention Localization for Zero-Shot Learning.

Yizhe Zhu, Jianwen Xie+3 more|arXiv (Cornell University)|Mar 1, 2019

Domain Adaptation and Few-Shot Learning14 references11 citations

TL;DR

This paper proposes a semantic-guided multi-attention localization model for zero-shot learning that automatically identifies discriminative object parts without human annotations. By jointly optimizing global and local features through embedding softmax and class-center triplet loss, the method achieves state-of-the-art performance on three benchmarks by enhancing inter-class dispersion and intra-class compactness.

ABSTRACT

Zero-shot learning extends the conventional object classification to the unseen class recognition by introducing semantic representations of classes. Existing approaches predominantly focus on learning the proper mapping function for visual-semantic embedding, while neglecting the effect of learning discriminative visual features. In this paper, we study the significance of the discriminative region localization. We propose a semantic-guided multi-attention localization model, which automatically discovers the most discriminative parts of objects for zero-shot learning without any human annotations. Our model jointly learns cooperative global and local features from the whole object as well as the detected parts to categorize objects based on semantic descriptions. Moreover, with the joint supervision of embedding softmax loss and class-center triplet loss, the model is encouraged to learn features with high inter-class dispersion and intra-class compactness. Through comprehensive experiments on three widely used zero-shot learning benchmarks, we show the efficacy of the multi-attention localization and our proposed approach improves the state-of-the-art results by a considerable margin.

Motivation & Objective

• To address the limitation of existing zero-shot learning methods that focus on visual-semantic mapping while neglecting discriminative feature learning.
• To automatically localize the most discriminative object parts for unseen classes without requiring human-annotated bounding boxes or part annotations.
• To improve zero-shot recognition performance by jointly learning global and local features with semantic supervision.
• To enhance feature quality by encouraging high inter-class dispersion and intra-class compactness through joint loss optimization.

Proposed method

• The model employs a multi-attention mechanism to automatically localize discriminative object parts from input images using only semantic descriptions.
• It jointly learns global features from the full image and local features from the detected parts to improve representation quality.
• The framework uses embedding softmax loss to align visual features with semantic embeddings.
• A class-center triplet loss is introduced to enforce inter-class separation and intra-class compactness in the feature space.
• The model is trained end-to-end with joint supervision from both loss functions to optimize for discriminative feature learning.
• Semantic representations of classes are used as supervision signals to guide attention localization and feature learning.

Experimental results

Research questions

• RQ1 Can automatic localization of discriminative object parts improve zero-shot recognition performance without human-annotated part locations?
• RQ2 How does joint optimization of global and local features with semantic supervision enhance feature discrimination in zero-shot learning?
• RQ3 To what extent does combining embedding softmax and class-center triplet loss improve inter-class separation and intra-class compactness?
• RQ4 How does the proposed method compare to state-of-the-art approaches on standard zero-shot learning benchmarks?

Key findings

• The proposed method achieves state-of-the-art performance on three widely used zero-shot learning benchmarks.
• The multi-attention localization mechanism successfully identifies discriminative parts without any human-annotated part annotations.
• The joint use of embedding softmax and class-center triplet loss leads to more compact intra-class features and better inter-class separation.
• The model demonstrates significant improvements over prior methods, with performance gains reported on standard zero-shot learning datasets.