[论文解读] Meta-learning for Few-shot Natural Language Processing: A Survey

Wenpeng Yin|arXiv (Cornell University)|Jul 19, 2020

Domain Adaptation and Few-Shot Learning参考文献 31被引用 63

一句话总结

对 Few-shot NLP 中元学习方法的综合综述，详细介绍基于度量的方法和基于优化的方法、数据集以及跨领域和跨任务的进展。

ABSTRACT

Few-shot natural language processing (NLP) refers to NLP tasks that are accompanied with merely a handful of labeled examples. This is a real-world challenge that an AI system must learn to handle. Usually we rely on collecting more auxiliary information or developing a more efficient learning algorithm. However, the general gradient-based optimization in high capacity models, if training from scratch, requires many parameter-updating steps over a large number of labeled examples to perform well (Snell et al., 2017). If the target task itself cannot provide more information, how about collecting more tasks equipped with rich annotations to help the model learning? The goal of meta-learning is to train a model on a variety of tasks with rich annotations, such that it can solve a new task using only a few labeled samples. The key idea is to train the model's initial parameters such that the model has maximal performance on a new task after the parameters have been updated through zero or a couple of gradient steps. There are already some surveys for meta-learning, such as (Vilalta and Drissi, 2002; Vanschoren, 2018; Hospedales et al., 2020). Nevertheless, this paper focuses on NLP domain, especially few-shot applications. We try to provide clearer definitions, progress summary and some common datasets of applying meta-learning to few-shot NLP.

研究动机与目标

• Clarify definitions and motivation for meta-learning in few-shot NLP.
• Summarize major meta-learning paradigms (metric-based and optimization-based) and how they apply to NLP.
• Provide progress overview across NLP tasks and domains with representative datasets.
• Review datasets used for few-shot NLP and highlight real-world challenges and benchmarks.

提出的方法

• Explain metric-based meta-learning with embedding functions and similarity metrics (e.g., Siamese, Matching Networks, Prototypical Networks, Relation Networks).
• Explain optimization-based meta-learning (e.g., MAML, FOMAML, Reptile) and how they learn rapid adaptation.
• Compare meta-learning with transfer learning and multi-task learning. 0
• Survey NLP-specific progress, including within-problem and cross-problem meta-learning scenarios.
• Summarize representative NLP datasets (FewRel, CLINC150, ARSC, SNIPS) and their roles in few-shot evaluation.

实验结果

研究问题

• RQ1 What meta-learning formulations are most effective for few-shot NLP tasks?
• RQ2 How do metric-based and optimization-based approaches differ in NLP performance and training complexity?
• RQ3 How does meta-learning transfer across domains and across different NLP tasks?
• RQ4 What datasets and benchmarks best reveal capabilities and limitations of few-shot NLP meta-learning?

主要发现

• Metric-based meta-learning relies on learned embeddings and distance metrics to classify with few examples.
• Optimization-based meta-learning (MAML, FOMAML, Reptile) optimizes for rapid adaptation with few gradient steps.
• Across NLP studies, meta-learning methods can outperform finetuned baselines and some multi-task approaches in few-shot settings.
• Few-shot NLP progress is reported across domain-specific sentiment and intent classification, relation classification, and cross-domain tasks.
• Datasets like FewRel and CLINC150 are commonly used for simulated few-shot evaluation, with calls for more realistic benchmarks.