[Paper Review] Online Continual Learning with Maximal Interfered Retrieval

Rahaf Aljundi, Eugene Belilovsky+5 more|arXiv (Cornell University)|Jan 1, 2019

Domain Adaptation and Few-Shot Learning206 citations

TL;DR

This paper proposes Maximal Interfered Retrieval (MIR), a novel online continual learning method that improves rehearsal by selectively retrieving samples from memory that are predicted to suffer the most negative impact from upcoming parameter updates. By prioritizing these high-interference examples during replay, MIR significantly reduces catastrophic forgetting and achieves state-of-the-art performance on standard benchmarks, outperforming random and standard replay baselines.

ABSTRACT

Continual learning, the setting where a learning agent is faced with a never-ending stream of data, continues to be a great challenge for modern machine learning systems. In particular the online or single-pass through the data setting has gained attention recently as a natural setting that is difficult to tackle. Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks. These approaches typically rely on randomly selecting samples from the replay memory or from a generative model, which is suboptimal. In this work, we consider a controlled sampling of memories for replay. We retrieve the samples which are most interfered, i.e. whose prediction will be most negatively impacted by the foreseen parameters update. We show a formulation for this sampling criterion in both the generative replay and the experience replay setting, producing consistent gains in performance and greatly reduced forgetting. We release an implementation of our method at https://github.com/optimass/Maximally_Interfered_Retrieval

Motivation & Objective

• To address catastrophic forgetting in online continual learning, where models face a continuous stream of data without revisiting past examples.
• To improve upon existing replay-based continual learning methods that rely on random or heuristic sampling of stored experiences.
• To develop a principled, dynamic sampling strategy that identifies and prioritizes the most vulnerable examples to interference.
• To demonstrate consistent performance gains across standard continual learning benchmarks using both experience replay and generative replay settings.

Proposed method

• The method formulates a retrieval criterion based on predicting the expected negative impact (interference) of future parameter updates on each stored sample's prediction.
• It computes interference scores for each memory sample by estimating how much the model's output for that sample will degrade after the next forward pass and weight update.
• In experience replay, the model retrieves the top-k samples with highest interference scores from a stored buffer of past examples.
• In generative replay, it retrieves the most interfered samples by generating candidates and selecting those with maximal predicted interference.
• The approach integrates seamlessly into online continual learning pipelines, requiring minimal architectural changes.
• The method is implemented and released publicly at https://github.com/optimass/Maximally_Interfered_Retrieval for reproducibility and community use.

Experimental results

Research questions

• RQ1 Can a dynamic, interference-aware sampling strategy improve rehearsal performance in online continual learning?
• RQ2 How does prioritizing samples with maximal predicted interference compare to random or uniform sampling in replay?
• RQ3 Does the proposed method reduce forgetting more effectively than existing replay-based approaches across different benchmarks?
• RQ4 Can the interference-based retrieval strategy be effectively applied in both experience replay and generative replay settings?
• RQ5 What is the impact of interference-based retrieval on model accuracy and stability over long, continuous learning streams?

Key findings

• The proposed Maximal Interfered Retrieval method achieves consistent performance gains over random and standard replay baselines across multiple continual learning benchmarks.
• By retrieving samples with the highest predicted interference, the method significantly reduces catastrophic forgetting during online learning.
• The approach outperforms existing state-of-the-art methods in both experience replay and generative replay settings.
• The method demonstrates robustness and effectiveness in the single-pass, online continual learning setting, which is known to be particularly challenging.
• The implementation is publicly released, enabling reproducibility and further research in interference-aware continual learning.