[Paper Review] Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System

Elahe Arani, Fahad Sarfraz+1 more|arXiv (Cornell University)|Jan 29, 2022

Memory Processes and Influences37 citations

TL;DR

CLS-ER is a dual-memory experience replay method inspired by the brain's complementary learning systems, achieving state-of-the-art continual learning performance without task boundaries or strong data distribution assumptions.

ABSTRACT

Humans excel at continually learning from an ever-changing environment whereas it remains a challenge for deep neural networks which exhibit catastrophic forgetting. The complementary learning system (CLS) theory suggests that the interplay between rapid instance-based learning and slow structured learning in the brain is crucial for accumulating and retaining knowledge. Here, we propose CLS-ER, a novel dual memory experience replay (ER) method which maintains short-term and long-term semantic memories that interact with the episodic memory. Our method employs an effective replay mechanism whereby new knowledge is acquired while aligning the decision boundaries with the semantic memories. CLS-ER does not utilize the task boundaries or make any assumption about the distribution of the data which makes it versatile and suited for "general continual learning". Our approach achieves state-of-the-art performance on standard benchmarks as well as more realistic general continual learning settings.

Motivation & Objective

• Motivate continual learning by addressing catastrophic forgetting in non-stationary data streams.
• Incorporate rapid (hippocampus-like) and slow (neocortex-like) learning through dual semantic memories.
• Develop a general replay-based framework that does not rely on task boundaries or strong distributional assumptions.
• Consolidate knowledge across tasks by aligning the working model with semantic memories during replay.

Proposed method

• Introduce CLS-ER with three memories: episodic (buffered samples), long-term semantic (stable EMA weights), and short-term semantic (plastic EMA weights).
• Maintain two EMA-based semantic memories by updating the plastic and stable models at different rates (rP > rS) and with decay parameters (αP, αS).
• Use reservoir sampling to manage a fixed episodic memory, ensuring the buffer matches the data stream distribution over time.
• During training, select replay logits from the semantic memories based on which model yields higher likelihood for the ground-truth class, enforcing a consistency loss in addition to standard cross-entropy.
• Update the working model by minimizing a combination of cross-entropy on current and memory samples and a mean squared error consistency loss with the selected semantic logits.
• Infer using the stable model to emphasize long-term knowledge and generalization.

Experimental results

Research questions

• RQ1 Can a dual memory system with fast and slow semantic memories improve knowledge consolidation in continual learning without relying on task boundaries?
• RQ2 Does aligning the working model's decision boundary with semantic memories yield better retention and smoother adaptation across tasks?
• RQ3 Are EMA-based semantic memories (short-term and long-term) effective for general incremental learning across diverse CL settings?
• RQ4 Does a reservoir-based episodic memory combined with a consistency loss outperform traditional replay baselines across Class-IL, Domain-IL, and General Incremental Learning scenarios?

Key findings

• CLS-ER achieves state-of-the-art performance across multiple continual learning benchmarks and settings, including Class-IL, Domain-IL, and GCIL.
• Using dual semantic memories (short-term/plastic and long-term/stable) with EMA-based updates improves consolidation and reduces forgetting relative to baselines like ER, DER++, and GEM.
• CLS-ER demonstrates convergence to flatter minima and better calibration, indicating robustness to perturbations and more reliable uncertainty estimates.
• The method shows robustness to memory budget variations and demonstrates strong performance in online CL-like settings (e.g., MNIST-based GCIL).
• Empirical analyses reveal CLS-ER mitigates recent-task bias and yields more uniform task probabilities across sequences.