[논문 리뷰] TRGP: Trust Region Gradient Projection for Continual Learning

Sen Lin, Li Yang외 2명|arXiv (Cornell University)|2022. 02. 07.

Domain Adaptation and Few-Shot Learning인용 수 22

한 줄 요약

TRGP은 가장 관련 있는 오래된 작업을 선택하기 위한 계층별 신뢰 영역과 frozen weights를 재사용하기 위한 스케일된 가중치 프로젝션을 도입하여 순차 학습에서 전달 지식 전달을 개선하고 망각을 줄인다.

ABSTRACT

Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.

연구 동기 및 목표

• Motivate the need to balance forward knowledge transfer and forgetting in non-expansion continual learning setups.
• Propose a layer-wise trust region to identify the most related old tasks for each layer of a network.
• Introduce scaled weight projection to reuse frozen weights from selected old tasks in the trust region.
• Jointly optimize scaling matrices and network parameters to improve learning of new tasks while avoiding forgetting.

제안 방법

• Define layer-wise subspaces S_j^l from old tasks using SVD-based representations.
• Compute gradient projections onto old task subspaces to form a trust region TR_t^l for each layer.
• Select top-K correlated old tasks based on gradient projection norms within the trust region.
• Introduce a scaling matrix Q_j,t^l to perform Proj_Sj^l^Q(W^{l}) and reuse old-task knowledge without overwriting it.
• Update the model by optimizing L with effective weights W_eff^l that combine projections from trust-region tasks and orthogonal directions to old-task subspaces.
• Construct task input subspaces S_j^l for each old task via SVD of representations and select bases by rank-k approximations and thresholding.

실험 결과

연구 질문

• RQ1 How can task correlations be effectively characterized to facilitate knowledge transfer in continual learning?
• RQ2 Can a trust-region mechanism improve forward knowledge transfer without increasing forgetting in non-expansion CL methods?
• RQ3 What is the impact of using scaled weight projections of correlated old-task subspaces on final performance across benchmarks?

주요 결과

• TRGP achieves higher final accuracy than state-of-the-art methods across multiple benchmarks (PMNIST, CIFAR-100 Split, 5-Datasets, MiniImageNet).
• TRGP attains average ACC improvements over GPM of 2.43%, 1.98%, and 1.37% on PMNIST, CIFAR-100 Split, and MiniImageNet, respectively, and 2.34% over HAT on 5-Dataset.
• TRGP demonstrates lower backward transfer (BWT) than several competitors, indicating reduced forgetting, e.g., BWT improvements of about 0.2% to 0.6% relative to baselines on cited datasets.
• The method shows universal improvement across all tasks and is particularly effective on harder tasks, with gains over GPM more pronounced on difficult tasks.