[Paper Review] La-MAML: Look-ahead Meta Learning for Continual Learning

Gunshi Gupta, Karmesh Yadav+1 more|arXiv (Cornell University)|Jul 27, 2020

Domain Adaptation and Few-Shot Learning30 references34 citations

TL;DR

La-MAML is an online meta-learning algorithm for continual learning that modulates per-parameter learning rates with a replay buffer to reduce forgetting and improve transfer.

ABSTRACT

The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. While meta-learning shows great potential for reducing interference between old and new tasks, the current training procedures tend to be either slow or offline, and sensitive to many hyper-parameters. In this work, we propose Look-ahead MAML (La-MAML), a fast optimisation-based meta-learning algorithm for online-continual learning, aided by a small episodic memory. Our proposed modulation of per-parameter learning rates in our meta-learning update allows us to draw connections to prior work on hypergradients and meta-descent. This provides a more flexible and efficient way to mitigate catastrophic forgetting compared to conventional prior-based methods. La-MAML achieves performance superior to other replay-based, prior-based and meta-learning based approaches for continual learning on real-world visual classification benchmarks. Source code can be found here: https://github.com/montrealrobotics/La-MAML

Motivation & Objective

• Motivate continual learning under non-stationary task streams with limited capacity and forgetting.
• Develop an online, fast meta-learning algorithm that uses replay to mitigate interference.
• Introduce Look-ahead MAML (La-MAML) with per-parameter learning-rate modulation to pace learning across tasks.
• Demonstrate scalability and robustness on realistic vision benchmarks against replay-based, prior-based and meta-learning baselines.

Proposed method

• Extend Continual-MAML (C-MAML) as a base online meta-learning method with a replay buffer.
• Introduce per-parameter learnable learning rates (LRs) updated asynchronously to modulate inner updates.
• Derive gradients showing how LR hypergradients reflect task-gradient alignment and interference.
• Optimize an online meta-objective (OML) via k-step MAML updates and a meta-loss evaluated on a mix of current and replay samples.
• Update network weights and LRs in an asynchronous, look-ahead fashion to trade off learning speed and forgetting.
• Empirically compare La-MAML with EWC, GEM, MER, iCARL, Sync and ER on standard CL benchmarks and scalable vision tasks.

Experimental results

Research questions

• RQ1 Can online meta-learning with replay achieve competitive or superior forgetting control compared to existing CL methods?
• RQ2 Do per-parameter, learnable learning rates (LRs) provide a flexible mechanism to balance stability and plasticity across tasks?
• RQ3 Does asynchronous LR modulation improve gradient alignment and reduce interference in continual learning?
• RQ4 How does La-MAML perform on both standard continual learning and efficient lifelong learning (LLL) setups on real-world vision datasets?

Key findings

• La-MAML often matches or outperforms replay-based, prior-based and meta-learning baselines across MNIST, CIFAR-100 and TinyImagenet benchmarks.
• Asynchronous per-parameter LR modulation plus look-ahead updates yields faster learning and stronger gradient alignment than baselines.
• La-MAML demonstrates robustness and scalability to realistic vision tasks with competitive retained accuracy and lower backward transfer/forgetting metrics.
• LR modulation provides improvements over fixed-learning-rate C-MAML and outperforms simple ER baselines, especially as task difficulty increases.
• Gradient alignment measurements show La-MAML variants achieve higher cosine similarity between replay and new-task gradients than ER, indicating better cross-task transfer.