[Paper Review] A Closer Look at Few-shot Classification

Wei-Yu Chen, Yen‐Cheng Liu+3 more|arXiv (Cornell University)|Apr 8, 2019

Domain Adaptation and Few-Shot Learning31 references188 citations

TL;DR

This paper provides a unified, fair comparison of representative few-shot classification methods, showing that deeper backbones reduce method gaps, a strong Baseline++ baseline, and a cross-domain evaluation revealing limits of current meta-learning approaches.

ABSTRACT

Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the \miniI and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

Motivation & Objective

• Provide a unified, fair benchmark for few-shot classification methods.
• Assess how backbone depth affects performance differences among methods.
• Evaluate whether simple baselines can match state-of-the-art meta-learning under standard and cross-domain settings.
• Explore cross-domain generalization and domain shift impacts on few-shot learning.

Proposed method

• Establish Baseline and Baseline++ as simple transfer-learning baselines with fixed feature extractors and trainable novel-class classifiers.
• Adopt cosine-distance based classifier (Baseline++) to reduce intra-class variation.
• Compare with representative meta-learning methods (MatchingNet, ProtoNet, RelationNet, MAML) under standardized settings.
• Evaluate performance across datasets (mini-ImageNet, CUB) and 1-/5-shot scenarios.
• Introduce a cross-domain setting mini-ImageNet to CUB to study domain shift effects.
• Provide public source code for reproducible comparisons.

Experimental results

Research questions

• RQ1 How do different few-shot classification methods perform when evaluated with a consistent backbone and training setup?
• RQ2 Does a deeper backbone reduce performance gaps among methods?
• RQ3 Can a simple distance-based Baseline++ approach match meta-learning methods on standard benchmarks?
• RQ4 How do domain shifts between base and novel classes affect few-shot learning performance?
• RQ5 What is the impact of additional adaptation steps for meta-learning methods under cross-domain conditions?

Key findings

• A Baseline++ distance-based classifier consistently improves Baseline and competes with state-of-the-art meta-learning on mini-ImageNet and CUB with a Conv-4 backbone.
• Deeper backbones dramatically reduce performance gaps among methods, especially on the CUB dataset, and can even make Baseline or Baseline++ competitive with meta-learners.
• Under cross-domain evaluation (mini-ImageNet → CUB), Baseline outperforms all meta-learning methods, highlighting limitations of current meta-learning in domain shift scenarios.
• Increasing backbone depth generally benefits ProtoNet and other methods when domain differences are larger, but results vary by dataset and shot setting.
• Further adaptation (fine-tuning) improves certain meta-learning methods, notably MatchingNet and MAML, more than ProtoNet in cross-domain settings.