[Paper Review] Using Hindsight to Anchor Past Knowledge in Continual Learning

Arslan Chaudhry, Albert Gordo+3 more|arXiv (Cornell University)|Feb 19, 2020

Domain Adaptation and Few-Shot Learning7 citations

TL;DR

This paper proposes Hindsight Anchor Learning (HAL), a bilevel optimization method that improves continual learning by learning representative 'anchor' data points per class to minimize forgetting. By simulating future forgetting in hindsight and enforcing invariant predictions on these anchors, HAL significantly boosts accuracy and reduces forgetting beyond standard experience replay across multiple benchmarks.

ABSTRACT

In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of experience replay, re-learning on past data stored in a small buffer known as episodic memory. In this work, we complement experience replay with a new objective that we call anchoring, where the learner uses bilevel optimization to update its knowledge on the current task, while keeping intact the predictions on some anchor points of past tasks. These anchor points are learned using gradient-based optimization to maximize forgetting, which is approximated by fine-tuning the currently trained model on the episodic memory of past tasks. Experiments on several supervised learning benchmarks for continual learning demonstrate that our approach improves the standard experience replay in terms of both accuracy and forgetting metrics and for various sizes of episodic memories.

Motivation & Objective

• Address catastrophic forgetting in continual learning, where neural networks rapidly lose previously acquired knowledge when exposed to new tasks.
• Overcome limitations of existing experience replay methods, which still suffer from significant forgetting despite storing past data.
• Develop a method that enhances memory-based continual learning by identifying and preserving predictions on critical, hard-to-forget examples from past tasks.
• Improve generalization and stability in continual learning by learning representative anchor points that maximize forgetting under future training.
• Achieve state-of-the-art performance on standard continual learning benchmarks by combining experience replay with a novel anchoring objective.

Proposed method

• Use bilevel optimization to jointly minimize the current task loss while keeping predictions invariant on learned anchor points from past tasks.
• Learn anchor points via gradient ascent in input space to maximize an approximation of the forgetting loss for the current task.
• Approximate forgetting by fine-tuning the current model on the episodic memory of past tasks and measuring performance drop on the anchor points.
• Construct one anchor per class per task, located near the classifier’s decision boundary where forgetting is most likely.
• Integrate the anchoring objective into experience replay, where the model is trained on current data and the episodic memory, with predictions on anchors preserved.
• Use a nested optimization scheme: the outer loop updates model parameters to minimize current loss, while the inner loop optimizes anchor locations to maximize expected forgetting.

Experimental results

Research questions

• RQ1 Can we improve experience replay in continual learning by identifying and preserving predictions on critical, hard-to-forget examples from past tasks?
• RQ2 Does learning anchor points in hindsight—by simulating future forgetting—lead to better generalization and reduced forgetting compared to standard replay?
• RQ3 How does the proposed anchoring mechanism perform across different episodic memory sizes and continual learning benchmarks?
• RQ4 Can bilevel optimization be effectively used to learn representative data points that enhance model stability without increasing memory overhead?
• RQ5 Does the method outperform existing regularization, modular, and memory-based continual learning approaches in terms of accuracy and forgetting metrics?

Key findings

• HAL achieves state-of-the-art performance on four standard continual learning benchmarks: MNIST permutations, MNIST rotations, split CIFAR-100, and split miniImageNet.
• HAL improves accuracy by up to 7.5% and reduces forgetting by nearly 23% compared to the experience replay baseline across all benchmarks.
• The performance gain of HAL is consistent across various episodic memory sizes, from 1 to 5 examples per class per task.
• Anchors are located near the classifier’s decision boundary, confirming their role as high-risk points for forgetting.
• The method significantly outperforms strong baselines including EWC, ICARL, VCL, AGEM, MER, and MIR in both accuracy and forgetting reduction.
• The ablation study confirms that the anchoring objective is essential, as removing it leads to performance degradation compared to standard experience replay.