[Paper Review] Marginal Replay vs Conditional Replay for Continual Learning

Timothée Lesort, Alexander Gepperth+2 more|arXiv (Cornell University)|Oct 29, 2018

Domain Adaptation and Few-Shot Learning37 references4 citations

TL;DR

This paper introduces conditional replay, a generative replay method for continual learning that generates samples and labels jointly by conditioning on class labels, avoiding error-prone label inference. It demonstrates that conditional replay outperforms marginal replay—especially under low sample generation constraints—by maintaining balanced class distributions and enabling constant-time complexity, making it more suitable for real-time applications with limited memory and compute.

ABSTRACT

We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal replay") that generates samples independently of their class and assigns labels in a separate step. The main improvement in conditional replay is that labels for generated samples need not be inferred, which reduces the margin for error in complex continual classification learning tasks. We demonstrate the effectiveness of this approach using novel and standard benchmarks constructed from MNIST and FashionMNIST data, and compare to the regularization-based \ extit{elastic weight consolidation} (EWC) method.

Motivation & Objective

• Address catastrophic forgetting in continual learning by developing a more robust replay mechanism.
• Overcome the limitations of marginal replay, which requires separate label prediction for generated samples, introducing potential error sources.
• Propose and evaluate conditional replay as a superior alternative that generates samples and labels together, conditioned on class.
• Design novel benchmarks using MNIST and FashionMNIST with disjoint, permutation, and rotation tasks to rigorously evaluate replay strategies.
• Demonstrate that conditional replay enables constant-time continual learning with manageable memory and computational overhead, suitable for real-world deployment.

Proposed method

• Propose conditional replay: a generative model that samples from a class-conditional distribution P(x|y), producing both images and labels simultaneously.
• Implement conditional replay using GANs and VAEs to compare generative model performance in continual learning settings.
• Compare against marginal replay, where samples are generated independently of class and labels are predicted post-hoc using a classifier.
• Use a fixed number of generated samples per task, with conditional replay allowing precise control over class balance.
• Apply both methods on standard and novel benchmarks (disjoint, permutation, rotation tasks) to evaluate generalization and forgetting.
• Train a DNN on sequential tasks, with experience replay via conditional or marginal generation to mitigate catastrophic forgetting.

Experimental results

Research questions

• RQ1 Does conditional replay reduce error from label inference compared to marginal replay in continual learning?
• RQ2 How does conditional replay perform relative to marginal replay when the number of generated samples is limited?
• RQ3 Can conditional replay maintain high accuracy while ensuring constant-time complexity in continual learning?
• RQ4 How effective is conditional replay on challenging benchmarks like disjoint-class continual learning tasks?
• RQ5 What is the impact of class imbalance in generated data on marginal replay performance, and how does conditional replay mitigate it?

Key findings

• Conditional replay outperforms marginal replay when generating fewer samples, particularly at low sample budgets (e.g., 0.1tN vs tN per task), due to better class distribution control.
• Marginal replay underperforms when generating fewer samples because of severe class imbalance, which degrades classification accuracy.
• Conditional replay maintains balanced class distributions in generated data, directly improving model generalization and reducing forgetting.
• The performance gap between conditional and marginal replay diminishes as the number of generated samples increases, with marginal replay eventually catching up.
• Conditional replay enables constant-time continual learning, as the number of generated samples does not need to grow over time, unlike marginal replay.
• The disjoint-class continual learning benchmark is particularly challenging for discriminative models but is effectively handled by conditional replay, demonstrating its robustness.