[論文レビュー] An Overview of Deep Learning Architectures in Few-Shot Learning Domain

Shruti Jadon, Aryan Jadon|arXiv (Cornell University)|Aug 12, 2020

Domain Adaptation and Few-Shot Learning被引用数 41

ひとこと要約

few-shot 学習の深層学習アプローチの調査で、データ拡張、メトリックベース、モデルベース、および最適化ベースの手法（Siamese ネットワーク、Matching Networks、MANN、Meta Networks、MAML、LSTM メタラーナー）を網羅する。

ABSTRACT

Since 2012, Deep learning has revolutionized Artificial Intelligence and has achieved state-of-the-art outcomes in different domains, ranging from Image Classification to Speech Generation. Though it has many potentials, our current architectures come with the pre-requisite of large amounts of data. Few-Shot Learning (also known as one-shot learning) is a sub-field of machine learning that aims to create such models that can learn the desired objective with less data, similar to how humans learn. In this paper, we have reviewed some of the well-known deep learning-based approaches towards few-shot learning. We have discussed the recent achievements, challenges, and possibilities of improvement of few-shot learning based deep learning architectures. Our aim for this paper is threefold: (i) Give a brief introduction to deep learning architectures for few-shot learning with pointers to core references. (ii) Indicate how deep learning has been applied to the low-data regime, from data preparation to model training. and, (iii) Provide a starting point for people interested in experimenting and perhaps contributing to the field of few-shot learning by pointing out some useful resources and open-source code. Our code is available at Github: https://github.com/shruti-jadon/Hands-on-One-Shot-Learning.

研究の動機と目的

• Provide a concise introduction to deep learning architectures for few-shot learning with references to core works.
• Explain how deep learning is applied in low-data regimes from data preparation to model training.
• Offer starting points, resources, and open-source code for researchers to experiment in few-shot learning.

提案手法

• Categorize few-shot DL approaches into data augmentation, metric-based, model-based, and optimization-based methods.
• Describe core architectures: Siamese networks, Matching Networks, Memory Augmented Neural Networks (MANN), Neural Turing Machines, Meta Networks, Model-Agnostic Meta-Learning (MAML), and LSTM-based meta-learners.
• Explain data processing and training procedures for each method, including pairwise data construction for Siamese networks and support/query setups for Matching Networks.
• Discuss memory-augmented architectures and attention/memory mechanisms enabling rapid generalization in few-shot tasks.
• Summarize optimization-based meta-learning strategies that aim to initialize or adapt models with few gradient steps.
• Provide pointers to datasets (e.g., Omniglot, Mini-Imagenet) and open-source resources for experimentation.

実験結果

リサーチクエスチョン

• RQ1 What architectures enable effective few-shot learning in deep models?
• RQ2 How do data augmentation, metric-based learning, model-based memory mechanisms, and optimization-based meta-learning contribute to performance under scarce data?
• RQ3 What are practical training and data-preparation considerations for few-shot learning systems?
• RQ4 What resources and open-source tools exist to start experimenting in few-shot learning?

主な発見

• The survey covers prominent architectures including Siamese networks, Matching Networks, and memory-augmented models for few-shot learning.
• It highlights data augmentation and metric learning as foundational techniques for reducing data requirements.
• Model-based approaches leverage external memory and attention to enable rapid generalization across tasks.
• Optimization-based meta-learning methods like MAML and LSTM-based learners aim to provide fast adaptation with few gradient steps.
• The work references standard few-shot benchmarks and provides practical guidance and code resources for researchers.