[Paper Review] Learning from Few Examples: A Summary of Approaches to Few-Shot Learning

Archit Parnami, Minwoo Lee|arXiv (Cornell University)|Mar 7, 2022

Domain Adaptation and Few-Shot Learning139 citations

TL;DR

This survey reviews meta-learning and non-meta-learning approaches to few-shot learning, categorizing methods into metric-based, optimization-based, and model-based paradigms, and discusses hybrid approaches and progress.

ABSTRACT

Few-Shot Learning refers to the problem of learning the underlying pattern in the data just from a few training samples. Requiring a large number of data samples, many deep learning solutions suffer from data hunger and extensively high computation time and resources. Furthermore, data is often not available due to not only the nature of the problem or privacy concerns but also the cost of data preparation. Data collection, preprocessing, and labeling are strenuous human tasks. Therefore, few-shot learning that could drastically reduce the turnaround time of building machine learning applications emerges as a low-cost solution. This survey paper comprises a representative list of recently proposed few-shot learning algorithms. Given the learning dynamics and characteristics, the approaches to few-shot learning problems are discussed in the perspectives of meta-learning, transfer learning, and hybrid approaches (i.e., different variations of the few-shot learning problem).

Motivation & Objective

• Motivate and define the few-shot learning problem and its practical relevance.
• Survey and categorize recent few-shot learning approaches from meta-learning, transfer learning, and hybrid perspectives.
• Summarize main families of methods and key design choices for small-data learning.
• Highlight datasets, progress, and open challenges in the field.

Proposed method

• Categorizes few-shot learning into meta-learning-based and non-meta-learning-based approaches.
• Within meta-learning, differentiates metric-based, optimization-based, and model-based methods, plus hybrids.
• Details representative algorithms (e.g., Siamese/Matching/Prototypical/Relation networks; MAML, LEO, Proto-MAML) and their training paradigms.
• Describes episodic training and task distributions as core components of meta-learning.
• Discusses task-adaptive and attention-based enhancements to improve discriminability in embeddings.

Experimental results

Research questions

• RQ1 What are the main meta-learning paradigms for few-shot classification and how do they differ in modeling P(y|x)?
• RQ2 How do metric-based, optimization-based, and model-based approaches address the challenge of learning from few examples?
• RQ3 What are the key hybrid strategies and extensions (hybrid/few-shot cross-domain, generalized FSL) that improve performance?
• RQ4 What datasets and benchmarks drive progress in few-shot learning, and what progress trends are observed?
• RQ5 What open challenges remain in few-shot learning for real-world deployment?

Key findings

• Meta-learning remains the dominant paradigm for few-shot learning, with three main branches: metric-based, optimization-based, and model-based methods.
• Metric-based methods use embedding functions and distance metrics, enabling episodic training on M-way K-shot tasks.
• Optimization-based methods learn how to optimize learners across tasks, often via outer/inner loop meta-training (e.g., MAML, LSTM meta-learners).
• Model-based methods leverage external memory or rapid adaptation mechanisms to support few-shot inference.
• Hybrid approaches combine multiple ideas (e.g., cross-task conditioning, task-adaptive representations) to boost performance.
• The survey aggregates common datasets (Omniglot, miniImageNet, FC100, tieredImageNet) and benchmarks for evaluation.