Skip to main content
QUICK REVIEW

[论文解读] BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation

Chengshu Li, Ruohan Zhang|arXiv (Cornell University)|Mar 14, 2024
Anomaly Detection Techniques and ApplicationsComputer Science被引用 6
一句话总结

BEHAVIOR-1K 引入一个以人为本的基准,涵盖 1,000 项日常活动,基于 50 个场景和 9,000+ 件对象,在 OmniGibson 中实现,以实现真实的物理与渲染,以研究长期目标的具身AI。本文分析基线 RL 方法以及仿真到现实的迁移差距。

ABSTRACT

We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with rich physical and semantic properties. The second is OMNIGIBSON, a novel simulation environment that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids. Our experiments indicate that the activities in BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both of which remain a challenge for even state-of-the-art robot learning solutions. To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we provide an initial study on transferring solutions learned with a mobile manipulator in a simulated apartment to its real-world counterpart. We hope that BEHAVIOR-1K's human-grounded nature, diversity, and realism make it valuable for embodied AI and robot learning research. Project website: https://behavior.stanford.edu.

研究动机与目标

  • 通过对 1,461 名参与者的调查,将活动置于人类需求之中,以识别高优先级的日常任务。
  • 创建一个多样化的数据集,包含 50 个场景和 9,000+ 标注对象,以支持 1,000 个生态上可信的活动。
  • 开发 OmniGibson,一个可实现刚体、可变形和流体交互的真实感仿真环境,用于具身AI研究。
  • 提供基线评估,揭示 BEHAVIOR-1K 中当前 RL 能力以及仿真到现实的迁移挑战。

提出的方法

  • 进行以人为本的调查,以按愿望将日常活动交给机器人来排序,并提取活动定义。
  • 使用 BDDL 逻辑组装 BEHAVIOR-1K 数据集,包含 1,000 项活动、50 个场景,以及 9,000+ 物体模型,并配以丰富的物理和语义注解。
  • 在 Nvidia Omniverse/PhysX 5 上开发 OmniGibson,以模拟流体、可变形物、织物、扩展对象状态以及现实任务实例化所需的转变规则。
  • 在 OmniGibson 中评估最先进的 RL 基线,包括视觉-运动控制(RL-VMC)和动作原语方法(RL-Prim, RL-Prim.Hist),在稀疏奖励下。
  • 通过将 BEHAVIOR-1K 场景映射到真实的 Tiago 移动操作臂来分析仿真到现实的迁移,并比较策略性能和故障模式。
Figure 1: Developing a Human-Centered Benchmark for Embodied AI. Left: human preference score over 2,090 activities, ranked based on a survey on 1,461 participants. The distribution indicates the high diversity of needs and preferences of humans that should be reflected in a comprehensive benchmark.
Figure 1: Developing a Human-Centered Benchmark for Embodied AI. Left: human preference score over 2,090 activities, ranked based on a survey on 1,461 participants. The distribution indicates the high diversity of needs and preferences of humans that should be reflected in a comprehensive benchmark.

实验结果

研究问题

  • RQ1普通公众最希望机器人完成的日常活动是什么,以及覆盖人类需求需要多么多样的场景/对象?
  • RQ2当前的具身AI方法是否能在真实感仿真中解决 BEHAVIOR-1K 的活动,主要瓶颈是什么?
  • RQ3当将策略从 OmniGibson 转移到真实移动操作臂时,BEHAVIOR-1K 的仿真到现实差距是什么?
  • RQ4哪些组件(感知、规划、抓取、动力学)在长期、操作密集任务中最限制性能?

主要发现

  • BEHAVIOR-1K 的活动是长期目标且需要复杂的操作,给当前 AI 系统带来挑战。
  • 带记忆的动作原语(RL-Prim.Hist)比端到端视觉-运动控制在更长任务上取得更高的成功率。
  • 抓取和精确的基于物理的执行至关重要;在训练中简化这些会显著改变评估性能。
  • OmniGibson 比竞争的模拟器具有更高的视觉真实感,能提供更可信的传感器输入,但在感知和执行差异方面引入了仿真到现实的差距。
  • 真实机器人研究显示感知与导航差异是主要的仿真到现实来源,尽管仿真很逼真,抓取在现实世界中仍具有较大难度。
  • 多样化、以人为本的活动与高保真仿真的结合,揭示了具身AI与机器人学习仍有大量改进空间。
Figure 2: Elements of BEHAVIOR-1K. Our benchmark comprises two elements: BEHAVIOR-1K Dataset and OmniGibson . Left: BEHAVIOR-1K Dataset includes 1,000 BDDL activity definitions (top left), 50 realistic and diverse scenes (top right), and 9,000+ objects with properties annotated in the knowledge base
Figure 2: Elements of BEHAVIOR-1K. Our benchmark comprises two elements: BEHAVIOR-1K Dataset and OmniGibson . Left: BEHAVIOR-1K Dataset includes 1,000 BDDL activity definitions (top left), 50 realistic and diverse scenes (top right), and 9,000+ objects with properties annotated in the knowledge base

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。