[Paper Review] Continual Learning and Catastrophic Forgetting

Gido M. van de Ven, Nicholas Soures+1 more|arXiv (Cornell University)|Mar 8, 2024

Cognitive Science and Education Research4 citations

TL;DR

This book chapter provides a comprehensive review of continual learning in deep neural networks, focusing on catastrophic forgetting—the rapid loss of previously learned knowledge when learning new tasks. It outlines six core computational approaches—replay, parameter regularization, functional regularization, optimization-based methods, context-dependent processing, and template-based classification—while emphasizing that overcoming forgetting alone is insufficient; efficient, adaptive, and resource-conscious learning is essential for true continual intelligence.

ABSTRACT

This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had learned before, a phenomenon known as catastrophic forgetting. Especially in the last decade, continual learning has become an extensively studied topic in deep learning. This book chapter reviews the insights that this field has generated.

Motivation & Objective

• To analyze the challenges of continual learning in deep neural networks, particularly catastrophic forgetting.
• To identify the limitations of current deep learning models in incrementally learning from non-stationary data streams.
• To bridge insights between deep learning and cognitive science by examining shared and divergent approaches to continual learning.
• To evaluate and compare different continual learning paradigms, including task-based vs. task-free and incremental learning settings.
• To advocate for a holistic evaluation framework that includes performance, diagnostics, and resource efficiency metrics.

Proposed method

• Classifies continual learning into task-based and task-free settings, and into task-, domain-, and class-incremental learning, treating these as orthogonal dimensions.
• Reviews six main computational strategies: (i) experience replay, (ii) parameter regularization (e.g., distillation), (iii) functional regularization (e.g., synaptic intelligence), (iv) optimization-based methods (e.g., learning rate scheduling), (v) context-dependent processing, and (vi) template-based classification.
• Proposes a unified framework to integrate task-based and task-free continual learning, enabling flexible and scalable model adaptation.
• Emphasizes the use of multi-dimensional evaluation metrics covering performance (e.g., accuracy), diagnostic analysis (e.g., forgetting rate), and resource efficiency (e.g., memory and compute usage).
• Highlights the importance of aligning deep learning methods with cognitive principles, such as controlled forgetting, to improve long-term learning sustainability.
• Encourages interdisciplinary collaboration between deep learning and cognitive science to inform both artificial intelligence and our understanding of biological intelligence.

Experimental results

Research questions

• RQ1 Why is catastrophic forgetting a fundamental obstacle to continual learning in deep neural networks?
• RQ2 How do different continual learning settings—task-based, task-free, class-, domain-, and task-incremental—differ in their challenges and requirements?
• RQ3 To what extent can computational strategies like replay or regularization prevent catastrophic forgetting while maintaining generalization and efficiency?
• RQ4 What role does controlled or intentional forgetting play in enabling long-term continual learning, and how can it be modeled in artificial systems?
• RQ5 How can insights from cognitive science and neuroscience inform the design of more biologically plausible and efficient continual learning algorithms?

Key findings

• Catastrophic forgetting remains the most prominent challenge in continual learning, where deep networks rapidly overwrite previously learned knowledge when exposed to new data.
• Replay, parameter regularization, and functional regularization are effective in mitigating forgetting but often require significant memory or computational overhead.
• Optimization-based approaches and context-dependent processing enable more efficient adaptation by modifying training dynamics rather than modifying the loss function directly.
• Template-based classification allows for zero-shot generalization across previously unpaired categories without requiring explicit learning of discriminative boundaries.
• The effectiveness of continual learning methods varies significantly across different settings, indicating that no single approach universally solves the problem.
• A comprehensive evaluation of continual learning requires more than accuracy—it must include diagnostics (e.g., forgetting rate) and resource efficiency metrics to reflect real-world deployment constraints.