[Paper Review] Meta-Learning for Semi-Supervised Few-Shot Classification

Mengye Ren, Eleni Triantafillou+6 more|arXiv (Cornell University)|Mar 2, 2018

Domain Adaptation and Few-Shot Learning21 references726 citations

TL;DR

The paper extends Prototypical Networks to semi-supervised few-shot learning by incorporating unlabeled data within episodes, proposing several refinements (soft k-means, distractor handling, masking) and showing consistent gains across Omniglot, mini-ImageNet, and tiered-ImageNet.

ABSTRACT

In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for a learning algorithm is defined and trained on episodes representing different classification problems, each with a small labeled training set and its corresponding test set. In this work, we advance this few-shot classification paradigm towards a scenario where unlabeled examples are also available within each episode. We consider two situations: one where all unlabeled examples are assumed to belong to the same set of classes as the labeled examples of the episode, as well as the more challenging situation where examples from other distractor classes are also provided. To address this paradigm, we propose novel extensions of Prototypical Networks (Snell et al., 2017) that are augmented with the ability to use unlabeled examples when producing prototypes. These models are trained in an end-to-end way on episodes, to learn to leverage the unlabeled examples successfully. We evaluate these methods on versions of the Omniglot and miniImageNet benchmarks, adapted to this new framework augmented with unlabeled examples. We also propose a new split of ImageNet, consisting of a large set of classes, with a hierarchical structure. Our experiments confirm that our Prototypical Networks can learn to improve their predictions due to unlabeled examples, much like a semi-supervised algorithm would.

Motivation & Objective

• Motivate and formalize semi-supervised few-shot learning within meta-learning episodes.
• Extend Prototypical Networks to leverage unlabeled data during both training and inference.
• Develop robust unlabeled-data refinements that handle distractors in the unlabeled pool.
• Evaluate on adapted Omniglot and ImageNet-based benchmarks, and introduce tiered ImageNet for hierarchical class splits.

Proposed method

• Represent episodes with labeled support set, unlabeled pool, and a query set within a meta-learning framework.
• Refine class prototypes using unlabeled data via (i) Soft k-means extension of Prototypical Networks, (ii) Soft k-means with a distractor cluster, and (iii) Masked Soft k-means that uses a learned masking to downweight potential distractors.
• Train end-to-end with the refined prototypes using the standard Prototypical Network loss, enabling the embedding to adapt to semi-supervised refinement.
• Adapt Omniglot, mini-ImageNet, and tiered-ImageNet with labeled/unlabeled splits per class and optional distractors; evaluate 1-shot and 5-shot accuracy with and without distractors.
• Provide publicly available code for reproducibility (GitHub link).

Experimental results

Research questions

• RQ1 Can unlabeled data within an episode improve few-shot classification when classes in the unlabeled set cover the target classes or include distractors?
• RQ2 Do semi-supervised refinements of prototypes learned during meta-training outperform purely supervised Prototypical Networks and naive semi-supervised inference?
• RQ3 Which semi-supervised refinement strategy (soft k-means variants with/distractors/masking) offers the best robustness and accuracy across datasets and shot settings?
• RQ4 How does the tiered ImageNet hierarchy affect few-shot semi-supervised learning and generalization to distinct test classes?

Key findings

• All proposed semi-supervised Prototypical Network variants outperform the purely supervised baseline on Omniglot, mini-ImageNet, and tiered-ImageNet in most settings.
• In non-distractor settings, at least one semi-supervised variant beats baselines across datasets and shot numbers, with no single model universally best.
• In distractor scenarios, Masked Soft k-Means shows the most robust performance, often achieving state-of-the-art among the evaluated methods.
• Increasing the unlabeled set size M improves test accuracy, indicating the models learn to leverage unlabeled data through meta-training.
• The study introduces tiered ImageNet as a large-scale, hierarchically structured benchmark for semi-supervised few-shot learning.