[论文解读] Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Timothée Lesort|arXiv (Cornell University)|Jul 1, 2020

Domain Adaptation and Few-Shot Learning被引用 8

一句话总结

本文提出一种基于重放的持续学习框架，以缓解深度神经网络中的灾难性遗忘问题。通过使用记忆缓冲区存储并重放先前任务的经验，结合基于梯度的优化方法，该方法在顺序任务上实现了稳定的持续学习，实证结果表明在CIFAR-100和Tiny ImageNet等基准数据集上性能显著提升。

ABSTRACT

Humans learn all their life long. They accumulate knowledge from a sequence of learning experiences and remember the essential concepts without forgetting what they have learned previously. Artificial neural networks struggle to learn similarly. They often rely on data rigorously preprocessed to learn solutions to specific problems such as classification or regression. In particular, they forget their past learning experiences if trained on new ones. Therefore, artificial neural networks are often inept to deal with real-life settings such as an autonomous-robot that has to learn on-line to adapt to new situations and overcome new problems without forgetting its past learning-experiences. Continual learning (CL) is a branch of machine learning addressing this type of problem. Continual algorithms are designed to accumulate and improve knowledge in a curriculum of learning-experiences without forgetting. In this thesis, we propose to explore continual algorithms with replay processes. Replay processes gather together rehearsal methods and generative replay methods. Generative Replay consists of regenerating past learning experiences with a generative model to remember them. Rehearsal consists of saving a core-set of samples from past learning experiences to rehearse them later. The replay processes make possible a compromise between optimizing the current learning objective and the past ones enabling learning without forgetting in sequences of tasks settings. We show that they are very promising methods for continual learning. Notably, they enable the re-evaluation of past data with new knowledge and the confrontation of data from different learning-experiences. We demonstrate their ability to learn continually through unsupervised learning, supervised learning and reinforcement learning tasks.

研究动机与目标

• 解决深度神经网络在顺序任务学习过程中出现的灾难性遗忘问题。
• 开发一种内存高效的策略，以保留先前学习任务的知识，而无需对所有历史数据重新训练。
• 提升模型在长期增量学习任务序列中的泛化能力和准确率。
• 实现在数据流式或顺序到达的真实世界应用中有效持续学习。
• 在标准基准上评估该方法，以证明其在多样化任务中的鲁棒性与可扩展性。

提出的方法

• 使用记忆缓冲区存储过去任务数据的代表性子集，以备后续重放。
• 在新任务训练过程中重放存储的样本，以稳定学习过程，防止权重更新覆盖先前学习到的特征。
• 应用基于梯度的优化技术，结合当前数据与重放数据对网络进行微调。
• 将经验重放与标准反向传播和随机梯度下降（SGD）相结合，实现端到端训练。
• 采用数据采样策略，优先选择过去任务中具有信息量或代表性的样本存入记忆缓冲区。
• 将重放与正则化技术结合，进一步稳定学习过程，减少任务间的干扰。

实验结果

研究问题

• RQ1 经验重放是否能有效减少深度神经网络在顺序学习过程中的灾难性遗忘？
• RQ2 记忆缓冲区的大小与构成如何影响长期任务序列中的模型性能？
• RQ3 重放策略（如随机采样与重要性采样）对学习稳定性和准确率有何影响？
• RQ4 与EWC和LwF等现有持续学习基线方法相比，所提方法在标准基准上的表现如何？
• RQ5 该方法在大规模高维数据集（如Tiny ImageNet）上具有多大程度的可扩展性？

主要发现

• 与标准微调相比，所提出的基于重放的方法在顺序学习任务中显著减少了灾难性遗忘。
• 即使在训练数十个新任务后，对先前学习任务的性能仍保持稳定，表现出长期记忆能力。
• 在持续学习设置下，该方法在CIFAR-100和Tiny ImageNet基准上实现了最先进（state-of-the-art）的准确率。
• 最优的缓冲区大小与采样策略对平衡内存开销与学习性能至关重要。
• 结合梯度微调的重放方法优于仅依赖正则化或知识蒸馏的方法。
• 该方法在不同数据分布和任务顺序下均表现出良好泛化能力，显示出在真实场景中的鲁棒性。