[Paper Review] AllenAct: A Framework for Embodied AI Research

Luca Weihs, Jordi Salvador+5 more|arXiv (Cornell University)|Aug 28, 2020

Reinforcement Learning in Robotics59 references44 citations

TL;DR

AllenAct is a modular Python/PyTorch framework that unifies environments, tasks, and training pipelines for Embodied AI research, enabling reproducible experiments across multiple environments and algorithms.

ABSTRACT

The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement learning and increased interest from the computer vision, NLP, and robotics communities. This growth has been facilitated by the creation of a large number of simulated environments (such as AI2-THOR, Habitat and CARLA), tasks (like point navigation, instruction following, and embodied question answering), and associated leaderboards. While this diversity has been beneficial and organic, it has also fragmented the community: a huge amount of effort is required to do something as simple as taking a model trained in one environment and testing it in another. This discourages good science. We introduce AllenAct, a modular and flexible learning framework designed with a focus on the unique requirements of Embodied AI research. AllenAct provides first-class support for a growing collection of embodied environments, tasks and algorithms, provides reproductions of state-of-the-art models and includes extensive documentation, tutorials, start-up code, and pre-trained models. We hope that our framework makes Embodied AI more accessible and encourages new researchers to join this exciting area. The framework can be accessed at: https://allenact.org/

Motivation & Objective

• Motivate a unified, modular framework to reduce fragmentation in Embodied AI (E-AI) research.
• Provide first-class support for multiple E-AI environments and tasks.
• Facilitate reproducible experiments with flexible training pipelines and pre-trained models.
• Lower ramp-up cost and encourage broader participation in E-AI research.

Proposed method

• Introduce abstractions: ActorCriticModel, Sensor, Environment, Task, TaskSampler, Losses, and TrainingPipeline.
• Decouple Environment and Task to allow flexible goal definitions and easy task swapping across environments.
• Support diverse training regimes including on-policy PPO/A2C, IL (imitation learning), DAgger, and offline training with flexible loss combinations.
• Provide flexible TrainingPipeline with sequential stages and optional teacher forcing for curriculum-style learning.
• Enable multi-agent support, visualization tools, and pre-trained models to boost reproducibility and debugging.

Experimental results

Research questions

• RQ1 Can a modular framework unify multiple embodied environments and tasks to streamline research workflows?
• RQ2 How far can a single training-pipeline abstraction support various learning signals (RL, IL, offline) and curriculum strategies?
• RQ3 What design choices improve reproducibility and cross-environment generalization in Embodied AI research?
• RQ4 Does decoupling Task from Environment facilitate easier addition of new tasks and environments?
• RQ5 Can multi-agent and visualization features accelerate method development and evaluation in E-AI?

Key findings

• AllenAct reproduces results across iTHOR, RoboTHOR, and Habitat for PointNav and ObjectNav tasks using DD-PPO.
• The framework supports GoToLocal in BabyAI showing IL/RL hybrids can achieve high test performance with relatively few steps.
• IL-to-RL pipelines (e.g., BC/DAgger followed by PPO) can boost performance, with adaptive IL-RL combining strategies performing best on MiniGrid tasks.
• Go beyond on-policy constraints with teacher forcing and offline supervision to broaden algorithmic possibilities.
• Multi-agent experiments (e.g., Stag Hunt) are reproducible and yield comparable rewards to prior work under similar settings.
• The framework includes pre-trained models and tutorials to aid reproducibility and onboarding.