[论文解读] An Overview of Deep Learning Architectures in Few-Shot Learning Domain

Shruti Jadon, Aryan Jadon|arXiv (Cornell University)|Aug 12, 2020

Domain Adaptation and Few-Shot Learning被引用 41

一句话总结

对少样本学习的深度学习方法综述，涵盖数据增强、基于度量、基于模型和基于优化的方法，如 Siamese 网络、Matching Networks、MANN、Meta Networks、MAML，以及基于 LSTM 的元学习器。

ABSTRACT

Since 2012, Deep learning has revolutionized Artificial Intelligence and has achieved state-of-the-art outcomes in different domains, ranging from Image Classification to Speech Generation. Though it has many potentials, our current architectures come with the pre-requisite of large amounts of data. Few-Shot Learning (also known as one-shot learning) is a sub-field of machine learning that aims to create such models that can learn the desired objective with less data, similar to how humans learn. In this paper, we have reviewed some of the well-known deep learning-based approaches towards few-shot learning. We have discussed the recent achievements, challenges, and possibilities of improvement of few-shot learning based deep learning architectures. Our aim for this paper is threefold: (i) Give a brief introduction to deep learning architectures for few-shot learning with pointers to core references. (ii) Indicate how deep learning has been applied to the low-data regime, from data preparation to model training. and, (iii) Provide a starting point for people interested in experimenting and perhaps contributing to the field of few-shot learning by pointing out some useful resources and open-source code. Our code is available at Github: https://github.com/shruti-jadon/Hands-on-One-Shot-Learning.

研究动机与目标

• 为少样本学习的深度学习架构提供简明介绍，并参考核心工作。
• 解释在数据较少的情境中深度学习是如何应用的，从数据准备到模型训练。
• 为研究者在少样本学习中提供起点、资源和开源代码以进行实验。

提出的方法

• 将少样本深度学习方法分为数据增强、基于度量、基于模型和基于优化的方法。
• 描述核心架构：Siamese networks、Matching Networks、Memory Augmented Neural Networks (MANN)、Neural Turing Machines、Meta Networks、Model-Agnostic Meta-Learning (MAML)、以及基于 LSTM 的元学习器。
• 解释每种方法的数据处理和训练流程，包括为 Siamese networks 构建成对数据以及为 Matching Networks 设置的 support/query。
• 讨论记忆增强架构及注意力/记忆机制，如何在少样本任务中实现快速泛化。
• 总结旨在通过少量梯度步骤初始化或自适应模型的基于优化的元学习策略。
• 提供数据集（例如 Omniglot、Mini-Imagenet）和用于实验的开源资源的指引。

实验结果

研究问题

• RQ1 哪些架构能够在深度模型中实现有效的少样本学习？
• RQ2 数据增强、基于度量的学习、基于模型的记忆机制和基于优化的元学习在数据稀缺下如何提升性能？
• RQ3 少样本学习系统在训练和数据准备方面的实际考量有哪些？
• RQ4 现有哪些资源和开源工具可用于开始在少样本学习中的实验？

主要发现

• 该综述涵盖了包括 Siamese networks、Matching Networks 和记忆增强模型在内的少样本学习的显著架构。
• 它强调数据增强和度量学习作为降低数据需求的基础技术。
• 基于模型的方法利用外部记忆和注意力来实现跨任务的快速泛化。
• 基于优化的元学习方法如 MAML 和基于 LSTM 的学习器旨在通过少量梯度步骤提供快速适应。
• 该工作参考了标准的少样本基准并为研究人员提供实际指南和代码资源。