[论文解读] A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks

Eden Belouadah, Adrian Popescu+1 more|arXiv (Cornell University)|Nov 3, 2020

Domain Adaptation and Few-Shot Learning参考文献 102被引用 199

一句话总结

本文分析固定大小/类别增量学习方法，提出一个统一框架，并显示在不同设置下没有单一算法支配，内存和增量状态大小对性能影响较大。

ABSTRACT

The ability of artificial agents to increment their capabilities when confronted with new data is an open challenge in artificial intelligence. The main challenge faced in such cases is catastrophic forgetting, i.e., the tendency of neural networks to underfit past data when new ones are ingested. A first group of approaches tackles forgetting by increasing deep model capacity to accommodate new knowledge. A second type of approaches fix the deep model size and introduce a mechanism whose objective is to ensure a good compromise between stability and plasticity of the model. While the first type of algorithms were compared thoroughly, this is not the case for methods which exploit a fixed size model. Here, we focus on the latter, place them in a common conceptual and experimental framework and propose the following contributions: (1) define six desirable properties of incremental learning algorithms and analyze them according to these properties, (2) introduce a unified formalization of the class-incremental learning problem, (3) propose a common evaluation framework which is more thorough than existing ones in terms of number of datasets, size of datasets, size of bounded memory and number of incremental states, (4) investigate the usefulness of herding for past exemplars selection, (5) provide experimental evidence that it is possible to obtain competitive performance without the use of knowledge distillation to tackle catastrophic forgetting and (6) facilitate reproducibility by integrating all tested methods in a common open-source repository. The main experimental finding is that none of the existing algorithms achieves the best results in all evaluated settings. Important differences arise notably if a bounded memory of past classes is allowed or not.

研究动机与目标

• 定义六个增量学习算法的期望属性并对其进行分析。
• 提供类别增量学习的统一形式化表述。
• 建立一个全面的多数据集评估框架，具有不同的内存和状态设置。
• 研究基于追踪（herding）的示例选择的有用性。
• 演示在没有知识蒸馏的情况下也能实现具有竞争力的性能，并通过开源仓库实现可重复性。

提出的方法

• 引入六个属性（复杂度、内存、准确性、时效性、可塑性、可扩展性）来评估增量学习方法。
• 用初始状态和 T-1 个增量状态对类别增量学习问题进行形式化。
• 开发一个包含多数据集、增量状态配置和内存大小的通用评估框架。
• 在方法之间实验基于追踪的示例选择以评估其影响。
• 通过将增量学习视为不均衡学习并使用偏置校正技术，证明在不进行蒸馏的情况下也能获得具有竞争力的结果。
• 将经测试的方法整合到一个统一的开源仓库中，输入/输出保持一致。

实验结果

研究问题

• RQ1 固定大小的类别增量学习算法在多样化的视觉任务和数据集上如何比较？
• RQ2 有限内存大小对不同增量学习算法的相对性能有何影响？
• RQ3 在内存约束下，基于示例追踪的选择对各增量学习方法是否有益？
• RQ4 是否可以在不进行知识蒸馏的情况下实现具有竞争力的性能，且在何种条件下？
• RQ5 提出的六个属性如何与实际使用场景下对增量学习在现实世界中的建议相关？

主要发现

• 没有单一的类别增量学习算法在所有评估设置中占优。
• 当内存大小或增量状态数量发生变化时，性能差异显著。
• 内存可用性对相对性能的影响比其他增量学习参数更大。
• 基于追踪的示例选择对所有测试的方法都带来一致的收益。
• 通过将增量学习建模为不均衡学习并应用偏置校正技术，可以在不进行蒸馏的情况下取得具备竞争力的结果。
• 作者提供了一个开源仓库，便于跨数据集和配置的可重复性。