[Paper Review] Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning

Danruo Deng, Guangyong Chen+3 more|arXiv (Cornell University)|Oct 9, 2021

Domain Adaptation and Few-Shot LearningComputer Science45 references24 citations

TL;DR

The paper introduces FS-DGPM, which combines Flattening Sharpness with Dynamic Gradient Projection Memory to mitigate forgetting and improve learning of new tasks in continual learning, by adaptively reweighting past-task gradient bases and regulating loss landscape flatness.

ABSTRACT

The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones. To address such the 'sensitivity-stability' dilemma, most previous efforts have been contributed to minimizing the empirical risk with different parameter regularization terms and episodic memory, but rarely exploring the usages of the weight loss landscape. In this paper, we investigate the relationship between the weight loss landscape and sensitivity-stability in the continual learning scenario, based on which, we propose a novel method, Flattening Sharpness for Dynamic Gradient Projection Memory (FS-DGPM). In particular, we introduce a soft weight to represent the importance of each basis representing past tasks in GPM, which can be adaptively learned during the learning process, so that less important bases can be dynamically released to improve the sensitivity of new skill learning. We further introduce Flattening Sharpness (FS) to reduce the generalization gap by explicitly regulating the flatness of the weight loss landscape of all seen tasks. As demonstrated empirically, our proposed method consistently outperforms baselines with the superior ability to learn new skills while alleviating forgetting effectively.

Motivation & Objective

• Investigate how the weight loss landscape relates to stability and sensitivity in continual learning.
• Characterize how flatter loss landscapes correlate with better continual learning performance.
• Develop a method to dynamically adjust the importance of gradient bases from past tasks to improve learning of new tasks.
• Extend GPM with mechanisms to flatten sharpness and adaptively release less important bases to balance forgetting and adaptation.

Proposed method

• Model the gradient subspace of past tasks with a bases matrix M and an importance matrix Λ.
• Compute adversarial perturbations v to evaluate and reduce sharpness in the current task by perturbing weights within the past-task subspace.
• Update Λ based on gradients of the current and memory data to dynamically release or tighten updates on each basis.
• Update weights w by projecting updates away from the past-task subspace to preserve old knowledge while accommodating new tasks.
• Recompute the basis M via SVD on replay data to keep the subspace representing past tasks up to date.
• Provide theoretical grounding via PAC-Bayes bounds linking flatness, loss, and continual learning performance.

Experimental results

Research questions

• RQ1 How does the weight loss landscape shape continual learning stability and sensitivity?
• RQ2 Can flatter loss landscapes and lower sharpness improve generalization and reduce forgetting in continual learning?
• RQ3 Can past-task gradient bases be adaptively weighted to improve learning of new tasks without compromising old tasks?
• RQ4 Does combining Flattening Sharpness with Dynamic Gradient Projection Memory (FS-DGPM) yield superior continual learning performance compared to existing methods?

Key findings

• FS-DGPM consistently outperforms baselines across PMNIST, CIFAR-100 Split, CIFAR-100 Superclass, and TinyImageNet in terms of average accuracy.
• The method achieves strong performance on old tasks while maintaining or improving learning speed and accuracy on new tasks, indicating improved sensitivity without sacrificing stability.
• Ablation studies show flattening sharpness and adaptive soft-weighting of bases contribute to performance gains over GPM and related variants.
• FS-DGPM reduces forgetting (BWT) and attains higher overall accuracy than competing methods across multiple benchmarks.
• The approach remains robust across single-head and multi-head settings, with favorable memory efficiency compared to some baselines.