[Paper Review] Layerwise Optimization by Gradient Decomposition for Continual Learning

Shixiang Tang, Dapeng Chen+3 more|arXiv (Cornell University)|May 17, 2021

Domain Adaptation and Few-Shot Learning57 references4 citations

TL;DR

This paper proposes a novel continual learning framework that decomposes gradients from episodic memory into shared and task-specific components, enforcing consistency with shared gradients and orthogonality to task-specific ones. By optimizing gradients layer-wise to mitigate magnitude imbalance, the method achieves state-of-the-art performance on multiple continual learning benchmarks, outperforming prior methods like GEM and S-GEM by up to 3.9% on Split Tiny ImageNet.

ABSTRACT

Deep neural networks achieve state-of-the-art and sometimes super-human performance across various domains. However, when learning tasks sequentially, the networks easily forget the knowledge of previous tasks, known as "catastrophic forgetting". To achieve the consistencies between the old tasks and the new task, one effective solution is to modify the gradient for update. Previous methods enforce independent gradient constraints for different tasks, while we consider these gradients contain complex information, and propose to leverage inter-task information by gradient decomposition. In particular, the gradient of an old task is decomposed into a part shared by all old tasks and a part specific to that task. The gradient for update should be close to the gradient of the new task, consistent with the gradients shared by all old tasks, and orthogonal to the space spanned by the gradients specific to the old tasks. In this way, our approach encourages common knowledge consolidation without impairing the task-specific knowledge. Furthermore, the optimization is performed for the gradients of each layer separately rather than the concatenation of all gradients as in previous works. This effectively avoids the influence of the magnitude variation of the gradients in different layers. Extensive experiments validate the effectiveness of both gradient-decomposed optimization and layer-wise updates. Our proposed method achieves state-of-the-art results on various benchmarks of continual learning.

Motivation & Objective

• Address catastrophic forgetting in continual learning by leveraging inter-task gradient information instead of treating gradients independently.
• Overcome the limitation of previous methods that concatenate gradients across layers, which can be dominated by high-magnitude layers.
• Improve knowledge retention by distinguishing shared knowledge (common to all old tasks) from task-specific knowledge (unique to individual tasks).
• Develop a layer-wise optimization strategy to decouple gradient updates per layer, reducing the influence of gradient magnitude variation.
• Achieve state-of-the-art performance across task-incremental and class-incremental continual learning settings.

Proposed method

• Decompose the gradient of each old task into a shared component (common to all old tasks) and a task-specific component (unique to that task).
• Enforce the update gradient to be non-negative in inner product with the shared gradient, promoting consolidation of common knowledge.
• Impose orthogonality constraints between the update gradient and the space spanned by all task-specific gradients, preserving task-specific knowledge.
• Apply Principal Component Analysis (PCA) to relax the strict orthogonality constraint, focusing on the most critical gradient directions.
• Implement layer-wise optimization by solving gradient constraints independently per layer, avoiding magnitude imbalance from concatenated gradients.
• Use episodic memory replay with gradient constraints to jointly optimize old and new task losses during continual training.

Experimental results

Research questions

• RQ1 Can inter-task gradient information be effectively disentangled to improve knowledge consolidation in continual learning?
• RQ2 Does enforcing orthogonality to task-specific gradient subspaces help preserve task-specific knowledge while minimizing interference?
• RQ3 Can layer-wise gradient optimization reduce the dominance of high-magnitude gradients and improve old task performance?
• RQ4 How does gradient decomposition compare to conventional gradient constraint methods like GEM or A-GEM in terms of forgetting and accuracy?
• RQ5 Can the proposed method generalize across different continual learning settings, including task-incremental and class-incremental scenarios?

Key findings

• On Split CIFAR100 with one epoch, the method improves accuracy from 65.8% (GEM) to 69.3%, a 3.5% gain.
• On Split Tiny ImageNet, the method achieves 38.1% accuracy in one epoch, a 3.9% improvement over GEM’s 34.2%.
• In the class-incremental setting on Split CIFAR100, the method achieves 65.3% accuracy, matching the state-of-the-art MUC method and outperforming GEM by 3%.
• The method consistently outperforms GEM, A-GEM, and S-GEM across all benchmarks, including MNIST Permutation and Split CIFAR10.
• Ablation studies confirm that both gradient decomposition and layer-wise optimization contribute significantly to performance gains.
• The layer-wise strategy increases efficiency in reducing old task losses, as shown by faster convergence and lower forgetting rates.