[Paper Review] CPR: Classifier-Projection Regularization for Continual Learning

Sungmin Cha, Hsiang Hsu+3 more|arXiv (Cornell University)|Jun 12, 2020

Domain Adaptation and Few-Shot Learning19 references32 citations

TL;DR

CPR adds a classifier-output entropy regularization term to existing regularization-based continual learning methods, interprets it as projecting outputs toward uniform, and empirically improves stability and plasticity across tasks.

ABSTRACT

We propose a general, yet simple patch that can be applied to existing regularization-based continual learning methods called classifier-projection regularization (CPR). Inspired by both recent results on neural networks with wide local minima and information theory, CPR adds an additional regularization term that maximizes the entropy of a classifier's output probability. We demonstrate that this additional term can be interpreted as a projection of the conditional probability given by a classifier's output to the uniform distribution. By applying the Pythagorean theorem for KL divergence, we then prove that this projection may (in theory) improve the performance of continual learning methods. In our extensive experimental results, we apply CPR to several state-of-the-art regularization-based continual learning methods and benchmark performance on popular image recognition datasets. Our results demonstrate that CPR indeed promotes a wide local minima and significantly improves both accuracy and plasticity while simultaneously mitigating the catastrophic forgetting of baseline continual learning methods. The codes and scripts for this work are available at https://github.com/csm9493/CPR_CL.

Motivation & Objective

• Motivate and address catastrophic forgetting in continual learning through wide local minima.
• Introduce a general regularization patch (CPR) that promotes entropy in classifier outputs.
• Provide theoretical interpretation of CPR as an information-projection toward the uniform distribution.
• Demonstrate CPR's effectiveness by applying it to multiple regularization-based CL methods on standard benchmarks.
• Show that CPR improves both stability (forgetting) and plasticity (forward transfer) across tasks and domains.

Proposed method

• Define CPR as a sum of a cross-entropy loss, a KL-divergence term driving outputs toward the uniform distribution, and a past-weights regularization term.
• Interpret CPR as an I-projection of the classifier output toward a convex set centered at the uniform distribution.
• Justify the approach with a KL-divergence projection framework and a Pythagorean-type relation for KL.
• Apply CPR to several baseline regularization-based CL methods (EWC, SI, MAS, RWalk, AGS-CL) and evaluate on multiple datasets.
• Use ablations and feature-map visualizations to analyze the role of CPR in promoting wide local minima and robustness.
• Extend evaluation to continual reinforcement learning experiments on Atari tasks using PPO with CPR.

Experimental results

Research questions

• RQ1 Does adding a classifier-projection regularization term improve continual learning performance over standard regularization-based methods?
• RQ2 How does CPR influence stability (forgetting) and plasticity (forward transfer) in sequential task learning?
• RQ3 Can CPR be interpreted as an information-projection onto a KL-ball around the uniform distribution, and does this provide theoretical justification for its effectiveness?
• RQ4 Is CPR effective across diverse datasets and learning domains (supervised and reinforcement learning) when combined with existing CL methods?

Key findings

• CPR consistently improves average accuracy across tested regularization-based CL methods and datasets.
• CPR reduces forgetting (stability) and also enhances plasticity (forward transfer) as measured by the provided metrics.
• Empirical analysis shows CPR yields a wider loss landscape (wide local minima) than baseline methods.
• CPR can be interpreted as projecting classifier outputs toward a uniform distribution within a KL-ball, offering a principled explanation for its gains.
• Ablation studies indicate applying CPR from the first task yields strong benefits for subsequent tasks.