[Paper Review] Reptile: a Scalable Metalearning Algorithm

Alex Nichol, John Schulman|arXiv (Cornell University)|Mar 8, 2018

Domain Adaptation and Few-Shot Learning18 references646 citations

TL;DR

Reptile is a simple metalearning algorithm that learns a model initialization by repeatedly fine-tuning on sampled tasks and moving the initialization toward the resulting weights. Unlike MAML, it avoids inner-loop gradient computation, enabling efficient adaptation in tasks requiring many optimization steps, and achieves strong performance on few-shot classification benchmarks.

ABSTRACT

This paper considers metalearning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We present a remarkably simple metalearning algorithm called Reptile, which learns a parameter initialization that can be fine-tuned quickly on a new task. Reptile works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. Unlike MAML, which also learns an initialization, Reptile doesn't require differentiating through the optimization process, making it more suitable for optimization problems where many update steps are required. We show that Reptile performs well on some well-established benchmarks for few-shot classification. We provide some theoretical analysis aimed at understanding why Reptile works.

Motivation & Objective

• To develop a scalable metalearning algorithm that enables fast adaptation to new tasks with minimal data.
• To address the computational inefficiency of MAML in scenarios requiring many optimization steps by avoiding differentiation through the inner optimization process.
• To explore whether a simpler, non-differentiable metalearning approach can match or exceed the performance of gradient-based methods like MAML.
• To provide theoretical insight into why the algorithm works, despite its simplicity.

Proposed method

• Reptile samples a task from a distribution of tasks and performs standard stochastic gradient descent on it.
• After training on a task, the algorithm updates the global initialization by moving it toward the task-specific weights using a small learning rate.
• The update rule is: θ ← θ + α(θ_task − θ), where α is a small step size and θ is the initialization.
• This process is repeated over many tasks, gradually shifting the initialization to a region that generalizes well across tasks.
• The method does not require backpropagation through the inner optimization loop, reducing memory and computational overhead.
• The algorithm is applicable to any differentiable model and supports various optimization schedules.

Experimental results

Research questions

• RQ1 Can a metalearning algorithm that avoids inner-loop gradient computation still achieve strong few-shot generalization performance?
• RQ2 How does Reptile's performance compare to MAML on standard few-shot classification benchmarks?
• RQ3 What is the theoretical basis for Reptile's ability to learn effective model initializations?
• RQ4 Does Reptile scale effectively to tasks requiring many optimization steps?

Key findings

• Reptile achieves competitive performance on standard few-shot learning benchmarks such as Omniglot and miniImageNet.
• The algorithm performs well even when tasks require many optimization steps, where MAML's computational cost becomes prohibitive.
• Theoretical analysis suggests that Reptile's update rule approximates a gradient step in the direction of minimizing the expected meta-loss.
• Reptile's simplicity and lack of inner-loop backpropagation make it more scalable and easier to implement than MAML.