[论文解读] Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges

Timothée Lesort, Vincenzo Lomonaco+4 more|arXiv (Cornell University)|Jun 29, 2019

Domain Adaptation and Few-Shot Learning参考文献 214被引用 474

一句话总结

一篇全面综述，定义持续学习（CL），提出正式框架、分类体系、评估问题和学习策略，聚焦机器人技术与跨域迁移。

ABSTRACT

Continual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective changes through time, or where all the training data and objective criteria are never available at once. The evolution of the learning process is modeled by a sequence of learning experiences where the goal is to be able to learn new skills all along the sequence without forgetting what has been previously learned. Continual learning also aims at the same time at optimizing the memory, the computation power and the speed during the learning process. An important challenge for machine learning is not necessarily finding solutions that work in the real world but rather finding stable algorithms that can learn in real world. Hence, the ideal approach would be tackling the real world in a embodied platform: an autonomous agent. Continual learning would then be effective in an autonomous agent or robot, which would learn autonomously through time about the external world, and incrementally develop a set of complex skills and knowledge. Robotic agents have to learn to adapt and interact with their environment using a continuous stream of observations. Some recent approaches aim at tackling continual learning for robotics, but most recent papers on continual learning only experiment approaches in simulation or with static datasets. Unfortunately, the evaluation of those algorithms does not provide insights on whether their solutions may help continual learning in the context of robotics. This paper aims at reviewing the existing state of the art of continual learning, summarizing existing benchmarks and metrics, and proposing a framework for presenting and evaluating both robotics and non robotics approaches in a way that makes transfer between both fields easier.

研究动机与目标

• 澄清持续学习（CL）的定义与范围及其与机器人技术的相关性。
• 提出一个正式框架，以标准化呈现和评估 CL 方法。
• 厘清术语并将 CL 与相关范式（强化学习 RL、监督/无监督学习）联系起来。
• 确定用于实现机器人与非机器人 CL 研究之间转移的基准、指标和评估问题。
• 突出机器人系统中 CL 的机遇、挑战及未来发展方向。

提出的方法

• 提出持续学习的正式框架，包括持续分布、任务和学习情景的定义。
• 介绍三种 CL 情景：Single-Incremental-Task (SIT)、Multi-Task (MT) 和 Multi-Incremental-Task (MIT)。
• 澄清术语及其与在线学习、少样本学习、Curriculum 学习、元学习、迁移学习和主动学习的关系。
• 提出一组评估问题（数据可用性、先验知识、内存/计算、监督、性能），用于标准化评估。
• 定义内存和计算约束，并讨论在实际 CL 部署中放宽这些约束的方式。

实验结果

研究问题

• RQ1 在动态环境中，哪些正式定义和结构最能描述持续学习？
• RQ2 如何将 CL 框架化，以实现跨机器人与非机器人领域的公平、可迁移评估？
• RQ3 哪些是与机器人相关的关键 CL 情景和数据/内容更新类型，它们如何影响算法？
• RQ4 应有哪些指标、基准和评估问题来指导现实世界机器人系统的 CL 研究？
• RQ5 在将 CL 应用于具身代理和发展/机器人学习时，会带来哪些机遇与挑战？

主要发现

• 该论文提供了一个用于 CL 的正式框架，定义持续分布、任务和训练集。
• 它将 SIT、MT 和 MIT 作为标准 CL 情景来对随时间的任务结构进行分类。
• 它澄清了相关概念（online learning、curriculum learning、meta-learning、transfer learning、active learning）及其与 CL 的关系。
• 它提出了一组全面的评估问题，用以评估 CL 中的数据使用、内存、计算、监督和性能。
• 它讨论了内容更新类型（New Instances、New Concepts、NIC）及对预训练和持续适应的影响。
• 它概述了一个内存/计算约束框架，并讨论了放宽以反映机器人领域实际部署的方式。