[Paper Review] Dark Experience for General Continual Learning: a Strong, Simple Baseline

Pietro Buzzega, M. Boschini+3 more|arXiv (Cornell University)|Apr 15, 2020

Domain Adaptation and Few-Shot LearningComputer Science272 citations

TL;DR

The paper introduces Dark Experience Replay (DER) and DER++, simple replay-based baselines that store logits from the optimization trajectory and use them with knowledge distillation to mitigate forgetting under General Continual Learning; DER/DER++ outperform many methods across standard CL benchmarks and a new MNIST-360 setting.

ABSTRACT

Continual Learning has inspired a plethora of approaches and evaluation settings; however, the majority of them overlooks the properties of a practical scenario, where the data stream cannot be shaped as a sequence of tasks and offline training is not viable. We work towards General Continual Learning (GCL), where task boundaries blur and the domain and class distributions shift either gradually or suddenly. We address it through mixing rehearsal with knowledge distillation and regularization; our simple baseline, Dark Experience Replay, matches the network's logits sampled throughout the optimization trajectory, thus promoting consistency with its past. By conducting an extensive analysis on both standard benchmarks and a novel GCL evaluation setting (MNIST-360), we show that such a seemingly simple baseline outperforms consolidated approaches and leverages limited resources. We further explore the generalization capabilities of our objective, showing its regularization being beneficial beyond mere performance.

Motivation & Objective

• Motivate and formalize General Continual Learning where task boundaries are blurred and online training prevails.
• Propose a simple baseline that combines rehearsal with knowledge distillation by replaying logits from the optimization trajectory.
• Demonstrate that DER and DER++ achieve state-of-the-art performance under GCL settings and analyze their regularization and calibration properties.

Proposed method

• Define DER as minimizing the current task loss plus an alpha-weighted KL/logit-distance term between current logits and stored past logits from a reservoir buffer.
• Store logits (z) rather than ground-truth labels in the replay buffer to distill past knowledge.
• Use reservoir sampling to maintain a task-agnostic memory buffer from the data stream.
• DER++ adds a second regularization term that also aligns past data with ground-truth guidance via a beta-weighted loss component.
• Argue that matching logits approximates matching pre-softmax responses, linking KL divergence to Euclidean differences in logits.
• Provide algorithms for DER and DER++ with practical training steps.

Experimental results

Research questions

• RQ1 Does a simple logit-based replay baseline (DER) generalize across the three primary CL settings (Task-IL, Class-IL, Domain-IL) without task boundaries?
• RQ2 How does DER compare to regularization-based and other rehearsal-based methods in terms of accuracy, calibration, and training efficiency?
• RQ3 Can a reservoir-based, task-agnostic memory buffer effectively support continual learning with no test-time task identifiers?
• RQ4 Does DER++ provide additional benefits by incorporating ground-truth guidance in the replay?
• RQ5 What is the behavior of DER/DER++ on a novel GCL benchmark MNIST-360 that combines abrupt and gradual distribution shifts?

Key findings

• DER and DER++ achieve state-of-the-art performance on multiple CL benchmarks, often outperforming rehearsal- and non-rehearsal baselines.
• DER converges to flatter minima and yields better calibration than ER and other baselines, indicating improved robustness and interpretability.
• Using logits from the optimization trajectory (not task-boundary outputs) with reservoir sampling improves memory usefulness and consistency across tasks.
• DER/DER++ show strong performance particularly in Domain-IL and Class-IL settings, where task boundaries are absent or non-discriminative.
• DER++ generally provides the best results among the proposed methods, especially when buffer sizes are constrained.
• MNIST-360 demonstrates that DER/DER++ are effective under a novel GCL regime with both abrupt and gradual distribution shifts.