[论文解读] AllenAct: A Framework for Embodied AI Research

Luca Weihs, Jordi Salvador+5 more|arXiv (Cornell University)|Aug 28, 2020

Reinforcement Learning in Robotics参考文献 59被引用 44

一句话总结

AllenAct 是一个模块化的 Python/PyTorch 框架，统一 Embodied AI 研究的环境、任务和训练流程，使在多个环境和算法上的可重复实验成为可能。

ABSTRACT

The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement learning and increased interest from the computer vision, NLP, and robotics communities. This growth has been facilitated by the creation of a large number of simulated environments (such as AI2-THOR, Habitat and CARLA), tasks (like point navigation, instruction following, and embodied question answering), and associated leaderboards. While this diversity has been beneficial and organic, it has also fragmented the community: a huge amount of effort is required to do something as simple as taking a model trained in one environment and testing it in another. This discourages good science. We introduce AllenAct, a modular and flexible learning framework designed with a focus on the unique requirements of Embodied AI research. AllenAct provides first-class support for a growing collection of embodied environments, tasks and algorithms, provides reproductions of state-of-the-art models and includes extensive documentation, tutorials, start-up code, and pre-trained models. We hope that our framework makes Embodied AI more accessible and encourages new researchers to join this exciting area. The framework can be accessed at: https://allenact.org/

研究动机与目标

• 推动建立统一、模块化框架，以减少 Embodied AI (E-AI) 研究中的碎片化现象。
• 为多种 E-AI 环境和任务提供一流支持。
• 通过灵活的训练管道和预训练模型，促进可重复实验。
• 降低上手成本，鼓励更广泛参与 E-AI 研究。

提出的方法

• 引入抽象层：ActorCriticModel、Sensor、Environment、Task、TaskSampler、Losses 和 TrainingPipeline。
• 实现 Environment 与 Task 的解耦，以便定义灵活的目标并在不同环境中实现任务替换。
• 支持多样的训练范式，包括 on-policy PPO/A2C、IL（ imitation learning ）、DAgger，以及采用灵活损失组合的离线训练。
• 提供具有顺序阶段和可选教师强制的灵活 TrainingPipeline，用于课程式学习。
• 支持多智能体、可视化工具，以及预训练模型，以提升可重复性和调试效率。

实验结果

研究问题

• RQ1 一个模块化框架能否统一多个 embodied 环境和任务，从而简化研究工作流？
• RQ2 单一训练管道抽象能在多大程度上支持不同的学习信号（ RL、IL、离线）和课程策略？
• RQ3 哪些设计选择能提升 Embodied AI 研究中的可重复性和跨环境泛化？
• RQ4 将 Task 与 Environment 解耦是否有利于更易添加新任务和新环境？
• RQ5 多智能体和可视化特性是否能加速 E-AI 的方法开发和评估？

主要发现

• AllenAct 在 iTHOR、RoboTHOR 和 Habitat 上对 PointNav 和 ObjectNav 任务使用 DD-PPO 复现实验结果。
• 该框架支持 BabyAI 的 GoToLocal，显示 IL/RL 混合可以在相对较少的步骤内达到较高的测试性能。
• IL-to-RL 流水线（如 BC/DAgger 后接 PPO）可以提升性能，适应性 IL-RL 结合策略在 MiniGrid 任务上表现最好。
• 超越 on-policy 限制，使用教师强制与离线监督以拓展算法可能性。
• 多智能体实验（如 Stag Hunt）具有可重复性，在相似设置下获得的回报与前人工作相当。
• 框架包含预训练模型和教程，以帮助提升可重复性与入门学习。