[论文解读] From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence

Nicholas Roy, Ingmar Posner+18 more|arXiv (Cornell University)|Oct 28, 2021

Reinforcement Learning in Robotics参考文献 185被引用 33

一句话总结

本论文认为具身智能需要与传统 ML 不同的学习方法，概述关键挑战、归纳偏置，以及使机器人学习稳健、安全和具泛化性的研究方向。它还提出利用受 System 1 与 System 2 推理启发的架构以及组成表示来弥合 System 1 与 System 2 的推理。

ABSTRACT

Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains. Consequently, the notion of applying learning methods to a particular problem set has become an established and valuable modus operandi to advance a particular field. In this article we argue that such an approach does not straightforwardly extended to robotics -- or to embodied intelligence more generally: systems which engage in a purposeful exchange of energy and information with a physical environment. In particular, the purview of embodied intelligent agents extends significantly beyond the typical considerations of main-stream machine learning approaches, which typically (i) do not consider operation under conditions significantly different from those encountered during training; (ii) do not consider the often substantial, long-lasting and potentially safety-critical nature of interactions during learning and deployment; (iii) do not require ready adaptation to novel tasks while at the same time (iv) effectively and efficiently curating and extending their models of the world through targeted and deliberate actions. In reality, therefore, these limitations result in learning-based systems which suffer from many of the same operational shortcomings as more traditional, engineering-based approaches when deployed on a robot outside a well defined, and often narrow operating envelope. Contrary to viewing embodied intelligence as another application domain for machine learning, here we argue that it is in fact a key driver for the advancement of machine learning technology. In this article our goal is to highlight challenges and opportunities that are specific to embodied intelligence and to propose research directions which may significantly advance the state-of-the-art in robot learning.

研究动机与目标

• 将具身智能定义为与物理环境的能量与信息交换，并说明它为何与标准 ML 设置不同。
• 识别具身智能代理在学习中的核心挑战，包括安全性、非平稳性和有限能量约束。
• 主张支持跨任务和跨环境泛化的归纳偏置与架构原则。
• 提出包括核心知识、分层抽象、组成表示以及形态感知学习等在内的研究方向。
• 讨论具身智能系统的评估与验证挑战。

提出的方法

• 回顾并综合来自机器人学、机器学习、认知科学及相关领域的观点，以界定具身学习的要求。
• 刻画具身智能代理特有的归纳偏置，并讨论它们对架构与学习的影响。
• 引入受 Dual Process Theory 启发的框架（System 1 和 System 2），作为将快速、学习的策略与深思熟虑的规划结合的蓝图。
• 探讨组成性、因果关系和物理启发先验作为结构性归纳偏置的作用。
• 主张元学习、课程设计和数据高效策略以应对非平稳、真实世界环境。

实验结果

研究问题

• RQ1 哪些归纳偏置使具身智能代理能够在不断变化的任务和环境中有效且鲁棒地学习？
• RQ2 具身智能的架构应如何在快速、本能式响应与较慢、深思熟虑的推理之间取得平衡？
• RQ3 哪些表示和组合结构有助于机器人领域的泛化与数据效率？
• RQ4 代理的形态如何影响学习，及将形态整合到学习流程的有效方法？
• RQ5 在非平稳的真实世界环境中运行的具身智能系统如何评估与验证？

主要发现

• 稳健的具身学习需要与行动与感知相关的归纳偏置，而不仅仅是标准 ML 偏好。
• 双过程理论为将快速、直观策略与较慢、深思熟虑的推理与规划结合的架构提供动力。
• 组成性表示和具因果意识的模型对于在非平稳环境中的泛化至关重要。
• 学习架构应整合多层次抽象，以在保持任务特定性能的同时管理计算需求。
• 形态学（传感器、执行器、能量约束）显著影响代理能够学习和做什么，需要整合设计与学习方法。
• 对具身学习者的评估与验证仍然是开放挑战，需要有原则性的框架。