[Paper Review] Incremental Classifier Learning with Generative Adversarial Networks

Yue Wu, Yinpeng Chen+6 more|arXiv (Cornell University)|Feb 2, 2018

Generative Adversarial Networks and Image SynthesisComputer Science20 references98 citations

TL;DR

Proposes an incremental classifier learning framework that combines a new loss function (cross-entropy + distillation), bias removal, and GAN-generated past data to mitigate catastrophic forgetting without relying on real past data. Evaluates on CIFAR-100, Flower-102, and MS-Celeb-1M-Base showing competitive to state-of-the-art results and advantages in privacy and scalability.

ABSTRACT

In this paper, we address the incremental classifier learning problem, which suffers from catastrophic forgetting. The main reason for catastrophic forgetting is that the past data are not available during learning. Typical approaches keep some exemplars for the past classes and use distillation regularization to retain the classification capability on the past classes and balance the past and new classes. However, there are four main problems with these approaches. First, the loss function is not efficient for classification. Second, there is unbalance problem between the past and new classes. Third, the size of pre-decided exemplars is usually limited and they might not be distinguishable from unseen new classes. Forth, the exemplars may not be allowed to be kept for a long time due to privacy regulations. To address these problems, we propose (a) a new loss function to combine the cross-entropy loss and distillation loss, (b) a simple way to estimate and remove the unbalance between the old and new classes , and (c) using Generative Adversarial Networks (GANs) to generate historical data and select representative exemplars during generation. We believe that the data generated by GANs have much less privacy issues than real images because GANs do not directly copy any real image patches. We evaluate the proposed method on CIFAR-100, Flower-102, and MS-Celeb-1M-Base datasets and extensive experiments demonstrate the effectiveness of our method.

Motivation & Objective

• Address catastrophic forgetting in incremental classifier learning when past data are unavailable.
• Develop a loss function that effectively combines cross-entropy and distillation losses on old exemplars and new data.
• Balance old and new classes via a simple bias removal mechanism estimable on validation data.
• Use GANs to generate historical data and select representative exemplars to replace real data while preserving privacy.

Proposed method

• Define incremental learning as extending a classifier from n to n+m classes with new labeled data X^m.
• Propose a loss L = lambda L_d + (1 - lambda) L_c combining distillation loss and cross-entropy loss across old exemplars and new data.
• Introduce a bias scalar beta to scale outputs of the new classes and estimate it on a validation set to remove old/new class bias.
• Train a GAN to model the old data: G(z) generates images labeled by f^n via the old classifier, selecting generated samples with max class probability above a threshold theta.
• Replace real exemplars with GAN-generated samples in the incremental learning setup and reuse the same loss function for training.

Experimental results

Research questions

• RQ1 Can a combined distillation and cross-entropy loss improve retention of old class knowledge while learning new classes?
• RQ2 Does a simple bias removal strategy effectively balance old and new classes in incremental learning?
• RQ3 Are GAN-generated past data effective substitutes for real exemplars in mitigating forgetting, while offering privacy advantages?
• RQ4 How do exemplar selection, loss balancing, and bias parameters affect incremental learning performance?
• RQ5 What is the performance gap between using GAN-generated data and real exemplars across datasets?

Key findings

• The proposed loss function with old exemplars and new data outperforms LwF and iCaRL on CIFAR-100, Flower-102, and MS-Celeb-1M-Base.
• Bias removal via a scalar beta consistently improves accuracy by reducing old/new class bias across validation and test sets.
• GAN-generated past data achieve competitive results, outperforming LwF without exemplars and approaching real-exemplar performance when tuned.
• Ours-Real (with real exemplars) yields gains over iCaRL across CIFAR-100, Flower-102, and MS-Celeb-1M-Base and remains robust across batch settings (P=2,5,10,20).
• In extreme batch settings, incremental learning with real exemplars closely matches batch training upper bounds, illustrating effectiveness of the proposed loss and bias strategy.