[Paper Review] Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning

Limeng Qiao, Yemin Shi+4 more|arXiv (Cornell University)|Oct 5, 2019

Domain Adaptation and Few-Shot LearningComputer Science36 references38 citations

TL;DR

TEAM learns a task-specific episodic-wise metric for each few-shot task by formulating adaptation as a semi-definite programming problem and using transductive inference, improving few-shot classification with a bi-directional similarity strategy.

ABSTRACT

Few-shot learning, which aims at extracting new concepts rapidly from extremely few examples of novel classes, has been featured into the meta-learning paradigm recently. Yet, the key challenge of how to learn a generalizable classifier with the capability of adapting to specific tasks with severely limited data still remains in this domain. To this end, we propose a Transductive Episodic-wise Adaptive Metric (TEAM) framework for few-shot learning, by integrating the meta-learning paradigm with both deep metric learning and transductive inference. With exploring the pairwise constraints and regularization prior within each task, we explicitly formulate the adaptation procedure into a standard semi-definite programming problem. By solving the problem with its closed-form solution on the fly with the setup of transduction, our approach efficiently tailors an episodic-wise metric for each task to adapt all features from a shared task-agnostic embedding space into a more discriminative task-specific metric space. Moreover, we further leverage an attention-based bi-directional similarity strategy for extracting the more robust relationship between queries and prototypes. Extensive experiments on three benchmark datasets show that our framework is superior to other existing approaches and achieves the state-of-the-art performance in the few-shot literature.

Motivation & Objective

• Motivate the challenge of learning a generalizable classifier that adapts to highly data-scarce tasks in few-shot learning.
• Propose a meta-learning framework (TEAM) that tailors an episodic-wise metric per task using transductive inference.
• Integrate pairwise constraints and a regularization prior into a solvable SDP formulation for task-specific metric learning.
• Enhance robustness with an attention-based bi-directional similarity between queries and prototypes and a task-level data augmentation technique.

Proposed method

• Embed inputs with a task-agnostic feature extractor trained over a pool of few-shot tasks.
• Formulate per-task metric adaptation as a convex optimization (SDP) with a pair-constrained loss and a regularization term.
• Derive a closed-form solution M_t^* = (M_0^{-1} + γ M̃ − γ λ Ĉ)^{-1} and augment it with a covariance term Σ_t to obtain M_t^†.
• Introduce a Bi-SIM strategy that computes both query-to-prototype and prototype-to-query similarities and combines them multiplicatively.
• Apply Task Internal Mixing (TIM) to synthesize augmented tasks by convex-combining samples within a task.
• Optionally leverage transductive BN or explicit transduction during testing for improved performance.

Experimental results

Research questions

• RQ1 How can we tailor a discriminative metric to each few-shot task rather than using a shared task-agnostic metric?
• RQ2 Can transductive inference jointly leverage the support and query sets to improve few-shot classification performance?
• RQ3 Does formulating the adaptation as a solvable SDP enable efficient, on-the-fly task-specific metric learning without heavy gradient-based updates?
• RQ4 Does a bi-directional similarity mechanism improve robustness of query-label predictions under extremely limited data?

Key findings

• TEAM achieves state-of-the-art or competitive results on three benchmarks (miniImageNet, CIFAR-100, and CUB) across 5-way 1-shot and 5-way 5-shot settings.
• On miniImageNet with a ConvNet backbone, TEAM improves 5-way 1-shot and 5-way 5-shot accuracies over the baseline by substantial margins (e.g., 4.89 percentage points and 3.33 percentage points, respectively).
• Compared to published state-of-the-art methods on miniImageNet, TEAM provides an absolute improvement of 1.06% on 1-shot and 2.18% on 5-shot (as reported in the paper).
• TEAM (with transduction) yields notable gains on CIFAR-100 and CUB in both 5-way 1-shot and 5-shot tasks, with consistent improvements over non-transductive baselines across backbones.