[Paper Review] Statistical Mechanical Analysis of Catastrophic Forgetting in Continual Learning with Teacher and Student Networks

Haruka Asanuma, Shiro Takagi+4 more|arXiv (Cornell University)|May 16, 2021

Domain Adaptation and Few-Shot LearningComputer Science17 references8 citations

TL;DR

This paper develops a statistical mechanical framework to theoretically analyze catastrophic forgetting in continual learning using a teacher-student neural network setup. It shows that forgetting is minimized when input distributions overlap little and target task relationships are highly similar, while also identifying a novel 'overshoot' phenomenon where performance on old tasks recovers after temporary forgetting during continued learning.

ABSTRACT

When a computational system continuously learns from an ever-changing environment, it rapidly forgets its past experiences. This phenomenon is called catastrophic forgetting. While a line of studies has been proposed with respect to avoiding catastrophic forgetting, most of the methods are based on intuitive insights into the phenomenon, and their performances have been evaluated by numerical experiments using benchmark datasets. Therefore, in this study, we provide the theoretical framework for analyzing catastrophic forgetting by using teacher-student learning. Teacher-student learning is a framework in which we introduce two neural networks: one neural network is a target function in supervised learning, and the other is a learning neural network. To analyze continual learning in the teacher-student framework, we introduce the similarity of the input distribution and the input-output relationship of the target functions as the similarity of tasks. In this theoretical framework, we also provide a qualitative understanding of how a single-layer linear learning neural network forgets tasks. Based on the analysis, we find that the network can avoid catastrophic forgetting when the similarity among input distributions is small and that of the input-output relationship of the target functions is large. The analysis also suggests that a system often exhibits a characteristic phenomenon called overshoot, which means that even if the learning network has once undergone catastrophic forgetting, it is possible that the network may perform reasonably well after further learning of the current task.

Motivation & Objective

• To establish a theoretical framework for analyzing catastrophic forgetting in continual learning, which is currently dominated by empirical and heuristic methods.
• To model task similarity through input distribution overlap and input-output relationship similarity (weight space similarity) in a teacher-student learning setup.
• To analytically derive conditions under which a student network avoids catastrophic forgetting during sequential task learning.
• To investigate the emergence of non-monotonic forgetting dynamics, such as overshoot, in the generalization error trajectory.

Proposed method

• The study employs a single-layer linear student network trained via stochastic gradient descent (SGD) on two sequential tasks.
• Teacher networks define the true input-output relationships for each task, with their weights used to compute weight space similarity via inner product.
• Input space similarity is quantified by the overlap of input data distributions within a shared subspace of the input space.
• Generalization error is analytically derived using statistical mechanics, assuming Gaussian-distributed inputs and isotropic weight initialization.
• The framework is extended to non-Gaussian inputs via the hidden manifold model, where intrinsic dimensionality replaces input dimensionality in influencing generalization.
• Theoretical analysis is validated through numerical simulations, particularly examining the impact of overparametrization and learning rate on forgetting dynamics.

Experimental results

Research questions

• RQ1 Under what conditions does a student network avoid catastrophic forgetting when learning two sequential tasks?
• RQ2 How do input distribution similarity and input-output relationship similarity (weight space similarity) jointly influence forgetting behavior?
• RQ3 What causes the overshoot phenomenon, where generalization error on a previous task increases temporarily before recovering?
• RQ4 How does the non-Gaussian nature of real-world data, modeled via a hidden manifold, alter the theoretical predictions of forgetting and recovery?

Key findings

• Catastrophic forgetting is minimized when the similarity between input distributions is low and the similarity between target task relationships (weight space similarity) is high.
• The overshoot phenomenon occurs when the learning rate is large, causing temporary performance degradation on old tasks before recovery through continued learning.
• Overparametrization increases generalization error in the student network, which may reduce apparent forgetting if the error was not initially minimized.
• In the hidden manifold model, intrinsic dimensionality becomes more relevant than input dimensionality, suggesting that manifold structure modulates forgetting dynamics.
• Theoretical results on generalization error convergence are qualitatively robust under non-Gaussian inputs, though the dynamics of overshoot may be amplified due to faster learning.