[論文レビュー] Meta-learning for Few-shot Natural Language Processing: A Survey

Wenpeng Yin|arXiv (Cornell University)|Jul 19, 2020

Domain Adaptation and Few-Shot Learning参考文献 31被引用数 63

ひとこと要約

Few-shot NLP に適用されたメタ学習手法の包括的な調査で、メトリックベースと最適化ベースのアプローチ、データセット、およびドメインとタスク間の進展を詳述する。

ABSTRACT

Few-shot natural language processing (NLP) refers to NLP tasks that are accompanied with merely a handful of labeled examples. This is a real-world challenge that an AI system must learn to handle. Usually we rely on collecting more auxiliary information or developing a more efficient learning algorithm. However, the general gradient-based optimization in high capacity models, if training from scratch, requires many parameter-updating steps over a large number of labeled examples to perform well (Snell et al., 2017). If the target task itself cannot provide more information, how about collecting more tasks equipped with rich annotations to help the model learning? The goal of meta-learning is to train a model on a variety of tasks with rich annotations, such that it can solve a new task using only a few labeled samples. The key idea is to train the model's initial parameters such that the model has maximal performance on a new task after the parameters have been updated through zero or a couple of gradient steps. There are already some surveys for meta-learning, such as (Vilalta and Drissi, 2002; Vanschoren, 2018; Hospedales et al., 2020). Nevertheless, this paper focuses on NLP domain, especially few-shot applications. We try to provide clearer definitions, progress summary and some common datasets of applying meta-learning to few-shot NLP.

研究の動機と目的

• Few-shot NLP におけるメタ学習の定義と動機を明確にする。
• 主要なメタ学習パラダイム（メトリックベースと最適化ベース）を要約し、それらがNLPにどのように適用されるかを説明する。
• 代表的なデータセットを用いて、NLPタスクとドメイン全体の進展の概要を提供する。
• few-shot NLP に用いられるデータセットをレビューし、実際の課題とベンチマークを強調する。

提案手法

• 埋め込み関数と類似度指標を用いたメトリックベースのメタ学習を解説する（例：Siamese、Matching Networks、Prototypical Networks、Relation Networks）。
• 最適化ベースのメタ学習（例：MAML、FOMAML、Reptile）を解説し、どのように迅速な適応を学習するかを説明する。
• メタ学習と転移学習およびマルチタスク学習を比較する。 0
• 問題内・問題間のメタ学習シナリオを含む、NLP特化の進展を概説する。
• 代表的なNLPデータセット（FewRel、CLINC150、ARSC、SNIPS）とFew-shot評価におけるそれらの役割を要約する。

実験結果

リサーチクエスチョン

• RQ1 What meta-learning formulations are most effective for few-shot NLP tasks?
• RQ2 How do metric-based and optimization-based approaches differ in NLP performance and training complexity?
• RQ3 How does meta-learning transfer across domains and across different NLP tasks?
• RQ4 What datasets and benchmarks best reveal capabilities and limitations of few-shot NLP meta-learning?

主な発見

• Metric-based meta-learning relies on learned embeddings and distance metrics to classify with few examples.
• Optimization-based meta-learning (MAML, FOMAML, Reptile) optimizes for rapid adaptation with few gradient steps.
• Across NLP studies, meta-learning methods can outperform finetuned baselines and some multi-task approaches in few-shot settings.
• Few-shot NLP progress is reported across domain-specific sentiment and intent classification, relation classification, and cross-domain tasks.
• Datasets like FewRel and CLINC150 are commonly used for simulated few-shot evaluation, with calls for more realistic benchmarks.