[Paper Review] Three scenarios for continual learning

Gido M. van de Ven, Andreas S. Tolias|Lirias (KU Leuven)|Apr 15, 2019

Domain Adaptation and Few-Shot Learning27 references556 citations

TL;DR

The paper defines three continual learning scenarios based on whether test-time task identity is provided and whether it must be inferred, and provides an extensive empirical comparison of methods across these scenarios using split and permuted MNIST.

ABSTRACT

Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe three continual learning scenarios based on whether at test time task identity is provided and--in case it is not--whether it must be inferred. Any sequence of well-defined tasks can be performed according to each scenario. Using the split and permuted MNIST task protocols, for each scenario we carry out an extensive comparison of recently proposed continual learning methods. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of how efficient different methods are. In particular, when task identity must be inferred (i.e., class incremental learning), we find that regularization-based approaches (e.g., elastic weight consolidation) fail and that replaying representations of previous experiences seems required for solving this scenario.

Motivation & Objective

• Motivate standardized evaluation for continual learning by clarifying how test-time task identity affects difficulty.
• Introduce three continual learning scenarios with increasing difficulty (Task-IL, Domain-IL, Class-IL).
• Provide a comprehensive empirical comparison of representative continual learning methods across these scenarios.
• Show that replay-based approaches are crucial when task identity must be inferred (Class-IL).

Proposed method

• Define three scenarios based on test-time task identity availability and inference requirement.
• Compare multiple continual learning methods (XdG, EWC, Online EWC, SI, LwF, DGR, DGR+distill, iCaRL) under each scenario.
• Use a single consistent network architecture (2 hidden layers for split MNIST, 2 hidden layers with 400/1000 units for split/permuted MNIST) and task-specific or shared output configurations per scenario.
• Employ split MNIST and permuted MNIST task protocols with clearly defined task boundaries.
• Evaluate performance with standardized training settings (Adam optimizer, specified iterations, memory budgets) and report mean ± SEM over 20 seeds.

Experimental results

Research questions

• RQ1 How do the three continual learning scenarios differ in difficulty when evaluating common methods?
• RQ2 Are regularization-based approaches sufficient when task identity must be inferred, or do replay-based strategies offer the necessary capability?
• RQ3 How does performance vary between split MNIST (two-task) and permuted MNIST (ten-task) protocols across scenarios?
• RQ4 Can task boundaries and replay mechanisms enable scalable continual learning across diverse task sequences?

Key findings

• Task-IL is the easiest scenario; most methods perform well when task identity is provided at test time.
• Domain-IL and Class-IL are substantially harder; regularization-based methods (EWC, Online EWC, SI) struggle and fail in Class-IL.
• Replay-based methods (LwF, DGR, DGR+distill, iCaRL) achieve strong performance in Domain-IL and Class-IL, often exceeding 90% accuracy on split MNIST tasks.
• LwF can be effective on split MNIST, but may underperform on permuted MNIST due to input distribution differences; generative replay (DGR) and distillation variants often provide robust performance across scenarios.
• iCaRL demonstrates strong performance when exemplars are available, illustrating the value of exemplar replay in continual learning.
• Across tasks, replay-based approaches consistently outperform regularization-only strategies when task identity must be inferred.