[论文解读] Unsupervised Learning via Meta-Learning

Kyle Hsu, Sergey Levine+1 more|arXiv (Cornell University)|Oct 4, 2018

Domain Adaptation and Few-Shot Learning参考文献 55被引用 128

一句话总结

tldr: 该论文提出 CACTUs，一种无监督元学习框架，通过嵌入和聚类从未标注数据自动构建类似监督的任务，从而在下游任务中无需任何标签即可实现有效的少样本学习。

ABSTRACT

A central goal of unsupervised learning is to acquire representations from unlabeled data or experience that can be used for more effective learning of downstream tasks from modest amounts of labeled data. Many prior unsupervised learning works aim to do so by developing proxy objectives based on reconstruction, disentanglement, prediction, and other metrics. Instead, we develop an unsupervised meta-learning method that explicitly optimizes for the ability to learn a variety of tasks from small amounts of data. To do so, we construct tasks from unlabeled data in an automatic way and run meta-learning over the constructed tasks. Surprisingly, we find that, when integrated with meta-learning, relatively simple task construction mechanisms, such as clustering embeddings, lead to good performance on a variety of downstream, human-specified tasks. Our experiments across four image datasets indicate that our unsupervised meta-learning approach acquires a learning algorithm without any labeled data that is applicable to a wide range of downstream classification tasks, improving upon the embedding learned by four prior unsupervised learning methods.

研究动机与目标

• Enable efficient downstream learning from unlabeled data by optimizing a learning procedure rather than a proxy unsupervised objective.
• Automatically construct structured learning tasks from unlabeled data using embeddings and clustering.
• 在多个图像数据集上证明能够迁移到多样化的人为设计的下游任务。
• Show that the learned learning procedure can outperform embedding-only baselines and approach supervised meta-learning under certain conditions.

提出的方法

• Input unlabeled dataset and learn embeddings with an unsupervised embedding method E.
• Generate multiple partitions of the embedding space by running k-means with random scaling to create diverse task distributions.
• Construct M-way, K-shot classification tasks from selected clusters to form meta-training tasks without labels.
• Apply meta-learning algorithms (MAML and ProtoNets) to learn a procedure F that quickly adapts to new tasks.
• Evaluate the learned learning procedure on downstream human-designed tasks (character recognition, object classification, facial attribute discrimination) across multiple datasets.
• Compare CACTUs-based meta-learning to embedding-based baselines and to oracle supervised meta-learning.

实验结果

研究问题

• RQ1 Does unsupervised meta-learning (CACTUs) yield a learning procedure that improves downstream task performance over embedding-based methods?
• RQ2 Is CACTUs effective across different embedding spaces and unsupervised learning methods?
• RQ3 Can the learned learning procedure transfer to a variety of downstream tasks and shot settings (1-shot to 50-shot)?
• RQ4 How does unsupervised CACTUs compare to supervised meta-learning with hand-designed task distributions (oracle)?

主要发现

• CACTUs-MAML consistently outperforms embedding-based baselines on downstream few-shot tasks across several datasets.
• CACTUs-ProtoNets also show improved performance, though ProtoNets may underperform in some settings when meta-training and testing shots are mismatched.
• CACTUs yields usable priors for a range of downstream tasks (character, object, and facial attribute tasks) without any labels during meta-training.
• The choice of embedding space affects performance, and better embedding methods correlate with stronger unsupervised meta-learning results.
• Non-random task construction (via clustering in embedding space) substantially outperforms random or pixel-space task construction, and over multiple partitions adds robustness.
• Performance gaps relative to an oracle supervised meta-learning setup depend on task difficulty and data overlap, with easier tasks showing smaller gaps.