[Paper Review] PILOT: A Pre-Trained Model-Based Continual Learning Toolbox

Hailong Sun, Da-Wei Zhou+2 more|arXiv (Cornell University)|Sep 13, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

PILOT is a PyTorch-based toolbox that enables class-incremental continual learning using pre-trained models (PTMs), integrating state-of-the-art PTM-based methods like L2P, DualPrompt, and CODA-Prompt alongside traditional CIL algorithms (e.g., FOSTER, DER, MEMO) adapted for PTMs. It demonstrates that re-implemented traditional methods often outperform dedicated prompt-based methods on benchmarks like CIFAR100 and ImageNet-R, establishing a unified benchmark for PTM-based continual learning.

ABSTRACT

While traditional machine learning can effectively tackle a wide range of problems, it primarily operates within a closed-world setting, which presents limitations when dealing with streaming data. As a solution, incremental learning emerges to address real-world scenarios involving new data's arrival. Recently, pre-training has made significant advancements and garnered the attention of numerous researchers. The strong performance of these pre-trained models (PTMs) presents a promising avenue for developing continual learning algorithms that can effectively adapt to real-world scenarios. Consequently, exploring the utilization of PTMs in incremental learning has become essential. This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT also fits typical class-incremental learning algorithms (e.g., DER, FOSTER, and MEMO) within the context of pre-trained models to evaluate their effectiveness.

Motivation & Objective

• To address the gap in continual learning toolboxes that support pre-trained models (PTMs) for class-incremental learning (CIL).
• To enable systematic evaluation and comparison of both PTM-based and traditional CIL algorithms within a unified framework.
• To facilitate research in continual learning by providing a standardized, extensible, and efficient toolbox for PTM-based CIL.
• To demonstrate that traditional CIL methods, when adapted to PTMs, can outperform specialized prompt-based methods on standard benchmarks.

Proposed method

• PILOT implements a model factory that instantiates CIL models with pre-trained vision transformers or vision-language models as backbones.
• It supports both exemplar-based and exemplar-free CIL algorithms, with configurable memory strategies (fixed memory per class or total memory size).
• The toolbox integrates state-of-the-art PTM-based CIL methods such as L2P, DualPrompt, and CODA-Prompt, which use learnable prompts to adapt the frozen PTM for new classes.
• Traditional CIL methods (e.g., FOSTER, DER, MEMO) are re-implemented within the PTM framework to allow direct comparison with prompt-based approaches.
• Hyperparameters such as batch size, optimizer settings, learning rate, and prompt-specific parameters (e.g., prompt_pool size, g_prompt_length) are configurable via JSON configuration files.
• The training loop processes data sequentially per task, updates the classifier incrementally, and evaluates performance on all seen classes after each task to measure catastrophic forgetting.

Experimental results

Research questions

• RQ1 Can traditional CIL algorithms, when adapted to pre-trained models, achieve better performance than dedicated prompt-based methods in class-incremental learning?
• RQ2 How do exemplar-based and exemplar-free strategies perform when applied to pre-trained models in continual learning?
• RQ3 What is the relative performance of state-of-the-art PTM-based CIL methods (e.g., L2P, CODA-Prompt) compared to re-implemented traditional methods on standard benchmarks?
• RQ4 To what extent does model initialization with a pre-trained model improve continual learning performance compared to random initialization?
• RQ5 How does the choice of pre-trained backbone (e.g., ViT, CLIP) affect the effectiveness of different CIL strategies?

Key findings

• On the 10-stage ImageNet-R benchmark, FOSTER with exemplars achieved an average accuracy (A̅) of 82.49 and final accuracy (AB) of 76.00, outperforming CODA-Prompt (A̅: 78.48, AB: 73.40) and ADAM w/ Adapter (A̅: 79.39, AB: 72.87).
• In the ImageNet-R setting, FOSTER’s average accuracy was 3.1 points higher than ADAM w/ Adapter and 2.6 points higher than CODA-Prompt.
• On CIFAR100, FOSTER achieved the highest final accuracy (87.92) among all methods tested, surpassing even the best prompt-based method, CODA-Prompt (86.85).
• Traditional methods like FOSTER and MEMO, when adapted to PTMs, consistently outperformed dedicated prompt-based methods in both average and final accuracy across benchmarks.
• The re-implementation of exemplar-based methods like iCaRL and DER within the PTM framework showed strong performance, indicating that exemplar mechanisms remain effective when combined with pre-trained features.
• Exemplar-free methods such as L2P and DualPrompt achieved competitive results, but still underperformed compared to exemplar-based FOSTER and MEMO on both CIFAR100 and ImageNet-R.