[Paper Review] Online Coreset Selection for Rehearsal-based Continual Learning

Jaehong Yoon, Divyam Madaan+2 more|arXiv (Cornell University)|Jun 2, 2021

Domain Adaptation and Few-Shot Learning46 references40 citations

TL;DR

OCS proposes online coreset selection for rehearsal-based continual learning using minibatch similarity, sample diversity, and coreset affinity to select representative and informative samples at each iteration, improving task adaptation while reducing forgetting.

ABSTRACT

A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a current dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.

Motivation & Objective

• Address how to select high-quality coresets in rehearsal-based continual learning under realistic data conditions (imbalance, noise).
• Develop an online, gradient-based coreset selection mechanism that favors current-task relevance while minimizing interference with past tasks.
• Provide a method that is simple, scalable, and compatible with existing rehearsal-based CL approaches.
• Demonstrate robustness across balanced, imbalanced, and noisy data regimes and show collaboration with baseline CL methods.

Proposed method

• Define three gradient-based selection criteria: minibatch similarity (S) to measure representativeness of a data point for the current minibatch,
• sample diversity (V) to reduce redundancy among selected samples, and
• coreset affinity (A) to minimize interference with gradients from past task knowledge.
• Select top-k samples per minibatch by maximizing S + V + tau*A, where tau controls plasticity vs. stability.
• Train on the selected coreset for the current task plus replay buffer samples, updating the model with a combined loss that balances current-task adaptation and past-task replay (Equation 9).
• Enable online coreset construction per minibatch and progressively memorize coresets to a replay buffer of fixed size (J) for continual learning.

Experimental results

Research questions

• RQ1 How can a coreset be selected online to maximize current-task adaptation while protecting past task knowledge in rehearsal-based CL?
• RQ2 Do gradient-based criteria (similarity, diversity, and affinity) yield representative, non-redundant coresets that reduce forgetting across realistic data streams?
• RQ3 Is the proposed Online Coreset Selection (OCS) approach compatible with various CL benchmarks and baselines, including imbalanced and noisy settings?

Key findings

• OCS outperforms strong rehearsal-based baselines across balanced, imbalanced, and noisy continual learning benchmarks in terms of average accuracy and forgetting.
• OCS demonstrates sample-efficient improvements by selectively curating the replay buffer rather than using random or uniform samples.
• OCS remains effective with varying coreset sizes and scales well with larger episodic memory budgets.
• OCS is compatible with existing CL methods and can collaboratively enhance task adaptation and mitigate forgetting.
• Ablation studies support the benefit of gradient-based criteria over raw-input baselines.