[Paper Review] Class-Incremental Learning: A Survey

Da-Wei Zhou, Qiwei Wang+4 more|arXiv (Cornell University)|Feb 7, 2023

Domain Adaptation and Few-Shot Learning237 references64 citations

TL;DR

This paper surveys deep class-incremental learning (CIL) methods, organizing them data-, model-, and algorithm-centric, with unified evaluations and a focus on memory-budget fair comparisons.

ABSTRACT

Deep models, e.g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world. However, novel classes emerge from time to time in our ever-changing world, requiring a learning system to acquire new knowledge continually. Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally and build a universal classifier among all seen classes. Correspondingly, when directly training the model with new class instances, a fatal problem occurs -- the model tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. There have been numerous efforts to tackle catastrophic forgetting in the machine learning community. In this paper, we survey comprehensively recent advances in class-incremental learning and summarize these methods from several aspects. We also provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms empirically. Furthermore, we notice that the current comparison protocol ignores the influence of memory budget in model storage, which may result in unfair comparison and biased results. Hence, we advocate fair comparison by aligning the memory budget in evaluation, as well as several memory-agnostic performance measures. The source code is available at https://github.com/zhoudw-zdw/CIL_Survey/

Motivation & Objective

• Define class-incremental learning (CIL) and formalize the problem setting and goals.
• Survey existing deep CIL methods across data-centric, model-centric, and algorithm-centric categories.
• Provide a unified, fair evaluation framework emphasizing memory budgets over multiple benchmarks.
• Analyze exemplar management (memory buffers) and discuss privacy implications.
• Offer insights to guide future CIL research and real-world deployment.

Proposed method

• Taxonomically categorize CIL methods into data-centric, model-centric, and algorithm-centric (with subcategories).
• Chronicle methods over time to show evolving research focus (Figure 3).
• Conduct a unified empirical evaluation of 16 methods on benchmark datasets (CIFAR100 and ImageNet100/1000) and discuss memory-budget effects.
• Highlight and advocate for memory-budget aligned evaluation and budget-agnostic performance measures.
• Discuss exemplar set management, herding for exemplar selection, and alternatives like direct vs. generative replay.
• Provide public code for reproducing evaluations at the referenced GitHub repository.

Experimental results

Research questions

• RQ1 What taxonomy best captures the landscape of deep CIL methods?
• RQ2 How do data-centric, model-centric, and algorithm-centric approaches compare under fair memory-budgeted evaluations?
• RQ3 What are the practical implications of memory budgets on CIL performance and experimental comparisons?
• RQ4 How do exemplar management strategies affect forgetting and overall accuracy across tasks?

Key findings

• The survey offers a comprehensive, taxonomic view of deep CIL methods, organized by data-, model-, and algorithm-centric families.
• It provides a unified evaluation of 16 methods across traditional CNN- and ViT-backed approaches on CIFAR100 and ImageNet100/1000.
• Memory budget is a critical factor in fair method comparisons, and the paper advocates aligned budgets and budget-agnostic metrics.
• Exemplar management and data replay/generative replay are central to addressing forgetting in CIL, with trade-offs in memory and privacy considerations.
• Recent trends include backbone/prompt expansion and ViT-based strategies that ease model growth under incremental tasks.
• The authors release source code to reproduce the evaluations.