[论文解读] Auto-conditioned Recurrent Mixture Density Networks for Learning Generalizable Robot Skills

Hejia Zhang, Eric Heiden+3 more|arXiv (Cornell University)|Sep 29, 2018

Robot Manipulation and Learning参考文献 34被引用 6

一句话总结

该论文提出了一种自条件循环混合密度网络（STM），通过建模状态转移并生成关节空间轨迹，从少量示范中学习可泛化的机器人操作技能。该方法能够在无需手工设计目标的情况下，实现对未见过的任务和比示范更长时域的快速泛化，在适应性和效率方面优于传统规划器。

ABSTRACT

Personal robots assisting humans must perform complex manipulation tasks that are typically difficult to specify in traditional motion planning pipelines, where multiple objectives must be met and the high-level context be taken into consideration. Learning from demonstration (LfD) provides a promising way to learn these kind of complex manipulation skills even from non-technical users. However, it is challenging for existing LfD methods to efficiently learn skills that can generalize to task specifications that are not covered by demonstrations. In this paper, we introduce a state transition model (STM) that generates joint-space trajectories by imitating motions from expert behavior. Given a few demonstrations, we show in real robot experiments that the learned STM can quickly generalize to unseen tasks and synthesize motions having longer time horizons than the expert trajectories. Compared to conventional motion planners, our approach enables the robot to accomplish complex behaviors from high-level instructions without laborious hand-engineering of planning objectives, while being able to adapt to changing goals during the skill execution. In conjunction with a trajectory optimizer, our STM can construct a high-quality skeleton of a trajectory that can be further improved in smoothness and precision. In combination with a learned inverse dynamics model, we additionally present results where the STM is used as a high-level planner. A video of our experiments is available at this https URL

研究动机与目标

• 解决在个人机器人领域中，仅从少量示范学习复杂且可泛化的操作技能的挑战。
• 克服传统运动规划流水线需繁琐手工设计目标的局限性。
• 使机器人能够泛化到示范数据中未出现的未见任务规范。
• 支持在技能执行过程中对变化目标的实时适应。
• 开发一种与轨迹优化和逆动力学模型集成的高层规划框架。

提出的方法

• 该方法采用状态转移模型（STM），通过从少量示范中模仿专家行为，学习生成关节空间轨迹。
• STM采用循环架构结合混合密度网络，对未来的状态和动作分布进行建模，实现具有不确定性感知的轨迹生成。
• 自条件机制根据当前任务上下文动态调整网络内部状态，提升对未见目标的泛化能力。
• STM与轨迹优化器结合，以提高生成运动骨架的平滑性和精度。
• 系统与学习得到的逆动力学模型集成，实现实世界机器人控制中的高层规划。
• 该框架支持在线适应，允许在执行过程中更改目标，而无需从头开始重新规划。

实验结果

研究问题

• RQ1 单一神经网络模型是否能仅通过少量专家示范泛化到未见的操作任务？
• RQ2 该模型在示范时域之外生成长时域轨迹的能力如何？
• RQ3 与传统规划器相比，该模型在执行过程中对变化目标的适应能力有多强？
• RQ4 STM与轨迹优化和逆动力学模型的集成如何提升最终运动质量？
• RQ5 STM能否在分层控制框架中作为有效的高层规划器？

主要发现

• STM成功泛化到示范数据中未出现的未见任务，展现出强大的零样本泛化能力。
• 该模型生成的关节空间轨迹时域长度超过专家示范，表明具备外推泛化能力。
• 系统在技能执行过程中能够实现实时目标适应，无需重新规划即可保持任务性能。
• 与轨迹优化器结合后，STM生成的运动更加平滑和精确，显著提升了最终轨迹质量。
• 与逆动力学模型的集成验证了STM在分层控制架构中作为高层规划器的可行性。