[Paper Review] Online Meta-Learning

Chelsea Finn, Aravind Rajeswaran+2 more|arXiv (Cornell University)|Feb 22, 2019

Domain Adaptation and Few-Shot Learning101 citations

TL;DR

The paper introduces an online meta-learning framework and the Follow The Meta Leader (FTML) algorithm, extending MAML to sequential tasks with an O(log T) regret guarantee and demonstrating strong empirical gains on vision tasks.

ABSTRACT

A central capability of intelligent systems is the ability to continuously build upon previous experiences to speed up and enhance learning of new tasks. Two distinct research paradigms have studied this question. Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch. In contrast, online (regret based) learning considers a sequential setting in which problems are revealed one after the other, but conventionally train only a single model without any task-specific adaptation. This work introduces an online meta-learning setting, which merges ideas from both the aforementioned paradigms to better capture the spirit and practice of continual lifelong learning. We propose the follow the meta leader algorithm which extends the MAML algorithm to this setting. Theoretically, this work provides an $\mathcal{O}(\log T)$ regret guarantee with only one additional higher order smoothness assumption in comparison to the standard online setting. Our experimental evaluation on three different large-scale tasks suggest that the proposed algorithm significantly outperforms alternatives based on traditional online learning approaches.

Motivation & Objective

• Motivate continual lifelong learning by unifying meta-learning and online learning concepts.
• Formulate the online meta-learning problem where tasks arrive sequentially and prior experience informs adaptation.
• Propose the Follow The Meta Leader (FTML) algorithm as an online meta-learning method.
• Provide theoretical regret guarantees and a practical deep-learning implementation for large-scale tasks.

Proposed method

• Define online meta-learning where updates U_t(w) are applied before evaluating on each task t.
• Propose FTML: w_{t+1} = argmin_w sum_{k=1}^t f_k(U_k(w)).
• Use a one-step gradient update U_t(w) = w - α ∇f̂_t(w) to obtain a MAML-like objective.
• Show that under standard smoothness and convexity assumptions, the composed function is convex, smooth, and leads to O(log T) regret.
• Provide a practical stochastic optimization instantiation inspired by MAML for deep networks (inner and outer loops, Grad/Adam updates).
• Demonstrate that FTML inherits regret guarantees comparable to Follow The Leader when the inner-update-augmented losses are suitably convex.

Experimental results

Research questions

• RQ1 Can online meta-learning be formulated to leverage past tasks for rapid adaptation to new tasks arriving sequentially?
• RQ2 Does FTML achieve sublinear regret against the best meta-learner in hindsight?
• RQ3 Can FTML be implemented effectively for deep neural networks on large-scale vision tasks?
• RQ4 How does online meta-learning compare to traditional online learning and joint training baselines in practical tasks?

Key findings

• FTML achieves an O(log T) regret bound against the best meta-learner in hindsight under stated assumptions.
• The MAML-like objective f_i(w - α ∇f̂_i(w)) is convex with appropriate step size, enabling efficient optimization.
• FTML provides empirical gains over Train On Everything (TOE) and joint training baselines, especially in data-efficient regimes.
• In Rainbow MNIST, FTML learns new tasks more efficiently with each added task, outperforming alternatives.
• In CIFAR-100, FTML enables faster task learning, with benefits from adapting all layers rather than only the last layer.
• In the sequential object pose prediction setting, FTML yields faster learning and better transfer than baseline methods.