[Paper Review] Learning Embedding Adaptation for Few-Shot Learning

Han-Jia Ye, Hexiang Hu+2 more|arXiv (Cornell University)|Dec 10, 2018

Domain Adaptation and Few-Shot Learning44 references72 citations

TL;DR

This paper proposes FEAT, a novel few-shot learning framework that adapts instance embeddings to the target classification task using a set-to-set Transformer-based function, enabling task-specific and discriminative representations. FEAT achieves state-of-the-art performance across standard and extended few-shot learning benchmarks, including cross-domain, transductive, generalized, and low-shot settings.

ABSTRACT

Learning with limited data is a key challenge for visual recognition. Many few-shot learning methods address this challenge by learning an instance embedding function from seen classes and apply the function to instances from unseen classes with limited labels. This style of transfer learning is task-agnostic: the embedding function is not learned optimally discriminative with respect to the unseen classes, where discerning among them leads to the target task. In this paper, we propose a novel approach to adapt the instance embeddings to the target classification task with a set-to-set function, yielding embeddings that are task-specific and are discriminative. We empirically investigated various instantiations of such set-to-set functions and observed the Transformer is most effective -- as it naturally satisfies key properties of our desired model. We denote this model as FEAT (few-shot embedding adaptation w/ Transformer) and validate it on both the standard few-shot classification benchmark and four extended few-shot learning settings with essential use cases, i.e., cross-domain, transductive, generalized few-shot learning, and low-shot learning. It archived consistent improvements over baseline models as well as previous methods and established the new state-of-the-art results on two benchmarks.

Motivation & Objective

• To address the limitation of task-agnostic embedding functions in few-shot learning, which are not optimized for discriminative performance on unseen classes.
• To develop a method that adapts pre-learned instance embeddings to be more discriminative for the specific target classification task.
• To explore effective set-to-set functions for embedding adaptation and identify the most suitable architecture for this task.
• To validate the proposed method across standard and extended few-shot learning settings, including cross-domain, transductive, generalized, and low-shot learning.
• To establish a new state-of-the-art performance on benchmark datasets in few-shot classification.

Proposed method

• The method introduces a set-to-set function that transforms a set of support embeddings into task-specific, discriminative representations.
• The Transformer architecture is selected as the set-to-set function due to its inherent ability to model long-range dependencies and relational inductive bias across embeddings.
• The model is trained end-to-end to adapt the embedding space such that embeddings from unseen classes are more separable for classification.
• The adaptation process is performed during inference, making the method flexible and applicable to various few-shot learning scenarios.
• The framework is evaluated on both standard few-shot classification and four extended settings, demonstrating robustness and generalization.
• The final model, named FEAT (Few-shot Embedding Adaptation with Transformer), integrates the embedding adaptation module into a standard few-shot learning pipeline.

Experimental results

Research questions

• RQ1 Can a set-to-set function effectively adapt pre-learned instance embeddings to improve discriminative performance on unseen classes in few-shot learning?
• RQ2 Which architecture is most effective for the set-to-set adaptation function, particularly in modeling relational and contextual information among support samples?
• RQ3 Does the proposed adaptation mechanism generalize across diverse few-shot learning settings, including cross-domain, transductive, generalized, and low-shot learning?
• RQ4 Can the method achieve state-of-the-art performance on standard and extended few-shot benchmarks?

Key findings

• FEAT achieves consistent improvements over baseline models and previous state-of-the-art methods across all evaluated few-shot learning benchmarks.
• The Transformer-based set-to-set function outperforms other instantiations, such as MLPs and RNNs, in terms of discriminative capability and generalization.
• The method establishes new state-of-the-art results on two major few-shot learning benchmarks, demonstrating superior performance in both standard and extended settings.
• The adaptation mechanism significantly enhances classification accuracy on unseen classes, particularly in challenging scenarios like cross-domain and low-shot learning.