[Paper Review] MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control

Nolan Wagener, Andrey Kolobov+4 more|arXiv (Cornell University)|Aug 15, 2022

Human Pose and Action Recognition4 citations

TL;DR

MoCapAct introduces a large-scale, high-quality dataset of expert policies that track 3.5 hours of MoCap data in the dm_control humanoid environment, enabling efficient learning of complex locomotion and motion completion. The dataset allows training of hierarchical policies and autoregressive GPT models that generalize across clips and generate natural, long-horizon motion from motion prompts.

ABSTRACT

Simulated humanoids are an appealing research domain due to their physical capabilities. Nonetheless, they are also challenging to control, as a policy must drive an unstable, discontinuous, and high-dimensional physical system. One widely studied approach is to utilize motion capture (MoCap) data to teach the humanoid agent low-level skills (e.g., standing, walking, and running) that can then be re-used to synthesize high-level behaviors. However, even with MoCap data, controlling simulated humanoids remains very hard, as MoCap data offers only kinematic information. Finding physical control inputs to realize the demonstrated motions requires computationally intensive methods like reinforcement learning. Thus, despite the publicly available MoCap data, its utility has been limited to institutions with large-scale compute. In this work, we dramatically lower the barrier for productive research on this topic by training and releasing high-quality agents that can track over three hours of MoCap data for a simulated humanoid in the dm_control physics-based environment. We release MoCapAct (Motion Capture with Actions), a dataset of these expert agents and their rollouts, which contain proprioceptive observations and actions. We demonstrate the utility of MoCapAct by using it to train a single hierarchical policy capable of tracking the entire MoCap dataset within dm_control and show the learned low-level component can be re-used to efficiently learn downstream high-level tasks. Finally, we use MoCapAct to train an autoregressive GPT model and show that it can control a simulated humanoid to perform natural motion completion given a motion prompt. Videos of the results and links to the code and dataset are available at https://microsoft.github.io/MoCapAct.

Motivation & Objective

• To lower the barrier to entry for research in simulated humanoid control by providing pre-trained expert policies for MoCap data tracking.
• To enable efficient learning of high-level behaviors by reusing low-level skills learned from MoCap data.
• To support generative motion completion by training a GPT-based policy on MoCapAct rollouts.
• To make high-quality MoCap-tracking policies and rollouts publicly available for broader research use.
• To demonstrate the utility of the dataset in training multi-clip policies and downstream tasks via reinforcement learning.

Proposed method

• Training deep reinforcement learning agents to track individual MoCap clips in the dm_control environment, producing expert policies for each clip.
• Collecting noise-injected rollouts (observations and actions) from each expert policy to form the MoCapAct dataset.
• Training a hierarchical policy using MoCapAct rollouts to track all clips simultaneously, with a shared low-level component.
• Fine-tuning the low-level component of the hierarchical policy on downstream tasks via reinforcement learning to enable fast adaptation.
• Training an autoregressive GPT model on 32-step sequences of observations and expert actions to perform motion completion from motion prompts.
• Evaluating GPT policies via rollout generation, measuring episode length and motion similarity using PCA projections.

Experimental results

Research questions

• RQ1 Can a single hierarchical policy be trained to track multiple MoCap clips using a shared low-level policy component?
• RQ2 Can the low-level policy learned from MoCapAct be effectively fine-tuned for new downstream tasks via reinforcement learning?
• RQ3 Can a GPT-based model generate long-horizon, natural humanoid motion from a short motion prompt using MoCapAct data?
• RQ4 How well does the GPT policy generalize to unseen clips, and what is the distribution of generated motion lengths?
• RQ5 To what extent do the generated motions match the kinematic patterns of the original MoCap clips?

Key findings

• The MoCapAct dataset contains expert policies that successfully track 3.5 hours of MoCap data across multiple clips in the dm_control environment.
• A single hierarchical policy trained on MoCapAct rollouts can track all considered MoCap clips, demonstrating effective multi-clip generalization.
• The low-level component of the hierarchical policy can be fine-tuned to learn new downstream tasks efficiently via reinforcement learning.
• The GPT-based policy generates motion completions with a mean episode length of 5.47 seconds (1.15× the prompt length), showing robust long-horizon generation.
• PCA projections show that GPT-generated actions often repeat demonstrated motions, though occasional deviations occur due to prompt ambiguity.
• The GPT model generalizes across diverse clips, including locomotion and non-locomotion motions, with consistent performance on both training and held-out validation sets.