[论文解读] Meta-Transfer Learning through Hard Tasks

Qianru Sun, Yaoyao Liu+3 more|arXiv (Cornell University)|Oct 7, 2019

Domain Adaptation and Few-Shot Learning被引用 4

一句话总结

该论文提出元迁移学习（MTL），一种新颖的元学习框架，通过为预训练深度神经网络（DNN）权重学习特定于任务的缩放与偏移函数，使DNN能够快速适应少样本任务。通过引入硬任务（HT）元小批量训练课程，MTL在miniImageNet、tieredImageNet和Fewshot-CIFAR100上实现了最先进性能，FC100上准确率最高提升1.85%，且仅需8,000个任务即可收敛——远少于MAML所需的240,000个任务。

ABSTRACT

Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, typical meta-learning models use shallow neural networks, thus limiting its effectiveness. In order to achieve top performance, some recent works tried to use the DNNs pre-trained on large-scale datasets but mostly in straight-forward manners, e.g., (1) taking their weights as a warm start of meta-training, and (2) freezing their convolutional layers as the feature extractor of base-learners. In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks. Specifically, meta refers to training multiple tasks, and transfer is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum that further boosts the learning efficiency of MTL. We conduct few-shot learning experiments and report top performance for five-class few-shot recognition tasks on three challenging benchmarks: miniImageNet, tieredImageNet and Fewshot-CIFAR100 (FC100). Extensive comparisons to related works validate that our MTL approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.

研究动机与目标

• 解决在使用深度神经网络（DNN）且标注数据有限时，少样本学习中过拟合的挑战。
• 克服现有元学习方法依赖浅层网络或DNN微调效率低下的局限性。
• 利用预训练DNN作为知识库，提升少样本泛化能力，而无需进行完整微调。
• 设计一种基于硬任务的高效训练课程，以加速收敛并提升性能。
• 证明MTL在不同网络架构和学习范式（包括监督与半监督少样本学习）中的泛化能力。

提出的方法

• 提出元迁移学习（MTL），通过学习预训练DNN权重的特定于任务的缩放与偏移函数，而非微调所有参数。
• 对特征提取器Θ和分类器θ均应用缩放与偏移（SS）操作，实现仅需极少可学习参数的高效适应。
• 引入硬任务（HT）元小批量方案，在元训练过程中优先选择具有挑战性的任务作为课程，提升学习效率。
• 采用两重优化训练元学习器：内层循环在少样本任务上适应基学习器，外层循环基于验证性能更新元参数。
• 使用预训练的ResNet-12、ResNet-18和ResNet-25作为骨干网络，元训练期间仅更新缩放与偏移参数。
• 通过将SS操作适配不同基学习器类型，将MTL框架推广至监督与半监督少样本学习。

实验结果

研究问题

• RQ1 在少样本学习中，基于预训练DNN的缩放与偏移函数进行元迁移学习，是否能优于标准微调和元学习基线方法？
• RQ2 与传统元小批量采样相比，所提出的硬任务（HT）元小批量课程是否能加速收敛并提升准确率？
• RQ3 MTL在不同DNN架构（如ResNet-12、ResNet-18、ResNet-25）的少样本设置下效果如何？
• RQ4 MTL是否能在监督与半监督少样本学习范式中持续提升性能？
• RQ5 HT元小批量方案在更具挑战性的基准（如FC100）上在多大程度上提升了学习效率？

主要发现

• MTL结合SS[Θ;θ]在miniImageNet、tieredImageNet和Fewshot-CIFAR100上达到最先进性能，在1-shot miniImageNet上相比无元学习方法准确率提升10.6%。
• HT元小批量方案在各基准上提升准确率0.6%–2.2%，在更具挑战性的FC100数据集上平均提升1.85%。
• MTL在miniImageNet上仅需约8,000个任务、在FC100上仅需约6,000个任务即可收敛，显著优于MAML的240,000个任务。
• 冻结预训练特征提取器Θ，仅通过SS对分类器θ进行元学习，相比完整微调在1-shot miniImageNet上性能提升约5%。
• HT元小批量在所有模型与基准上均持续提升验证性能，其中在FC100上提升最大。
• MTL框架泛化能力强：在经典监督与最先进半监督少样本学习模型上均显著提升性能。