[Paper Review] Gradient Episodic Memory for Continual Learning

David López-Paz, Marc’Aurelio Ranzato|arXiv (Cornell University)|Jun 26, 2017

Domain Adaptation and Few-Shot LearningComputer Science5 references497 citations

TL;DR

GEM introduces an episodic memory-based approach to continual learning that reduces forgetting and enables beneficial knowledge transfer across tasks, demonstrating strong performance on MNIST and CIFAR-100 variants.

ABSTRACT

One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.

Motivation & Objective

• Formalize continual learning with a sequence of tasks and non-iid data observations.
• Introduce metrics for forgetting (backward transfer) and forward transfer, plus average accuracy.
• Propose Gradient Episodic Memory (GEM) to minimize forgetting while allowing positive backward transfer.
• Evaluate GEM against state-of-the-art baselines on MNIST permutations, MNIST rotations, and incremental CIFAR-100.

Proposed method

• Maintain an episodic memory per task with a fixed budget to store representative examples.
• Formulate learning as minimizing the current loss subject to inequality constraints on past-task losses.
• Transform constraints into a quadratic program and solve its dual to obtain a projected gradient update.
• Allow positive backward transfer by not distilling past task predictions, only constraining their loss not to increase.
• Provide an algorithm that updates parameters via the projected gradient to avoid increasing past-task losses.

Experimental results

Research questions

• RQ1 How can continual learning be formalized to capture non-iid data streams and task transitions?
• RQ2 Can an episodic memory and gradient projection prevent forgetting while enabling transfer across tasks?
• RQ3 What is the impact of memory size and task order on forgetting and transfer in GEM?
• RQ4 How does GEM compare to regularization and memory-based baselines on standard continual learning benchmarks?

Key findings

• GEM achieves comparable or superior average accuracy across tasks compared to state-of-the-art baselines.
• GEM minimizes backward transfer (forgetting) and exhibits negligible or positive forward transfer.
• GEM scales efficiently by reducing optimization to a problem with variables equal to the number of tasks rather than the number of parameters.
• GEM exhibits minimal forgetting on CIFAR-100 and favorable backward transfer in experiments.
• GEM’s performance approaches that of iid multi-task training on benchmark datasets.