[Paper Review] Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors

Steven Bohez, Saran Tunyasuvunakool+19 more|arXiv (Cornell University)|Mar 31, 2022

Robotic Locomotion and ControlEngineering20 citations

TL;DR

The paper learns a reusable motor-skill module from MoCap data of humans and dogs to control legged robots, enabling zero-shot imitation and downstream tasks like controllable walking and ball dribbling with simulation-to-real transfer.

ABSTRACT

We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks. Importantly, due to the prior imposed by the MoCap data, our approach does not require extensive reward engineering to produce sensible and natural looking behavior at the time of reuse. This makes it easy to create well-regularized, task-oriented controllers that are suitable for deployment on real robots. We demonstrate how our skill module can be used for imitation, and train controllable walking and ball dribbling policies for both the ANYmal quadruped and OP3 humanoid. These policies are then deployed on hardware via zero-shot simulation-to-reality transfer. Accompanying videos are available at https://bit.ly/robot-npmp.

Motivation & Objective

• Motivate the use of human/animal movement priors to improve legged robot locomotion.
• Develop a general skill module trained from MoCap data that can be reused across tasks.
• Enable zero-shot transfer from simulation to real robots by leveraging regularized, natural-looking motions.
• Demonstrate downstream task performance including controllable walking and ball dribbling on two robot platforms.

Proposed method

• Retarget MoCap data from humans or dogs to two robots (ANYmal quadruped and OP3 humanoid).
• Train a universal goal-conditioned imitation policy with an encoder that maps future trajectory frames to a latent skill, and a decoder that maps the latent skill to joint commands.
• Impose an information bottleneck and regularize latent space toward a Gaussian AR(1) prior to obtain a compact, reusable skill space.
• Reuse the decoder as a low-level skill module by training a task-specific policy to output latent actions for downstream tasks.
• Train entirely in simulation with domain and dynamics randomization to enable zero-shot sim-to-real transfer.
• Evaluate both zero-shot imitation and downstream tasks in simulation and on hardware.

Experimental results

Research questions

• RQ1 Can MoCap-derived priors serve as a generalizable prior for legged robot control?
• RQ2 Can a single skill module imitate a wide range of MoCap trajectories and be reused for diverse tasks?
• RQ3 Does simulation-trained skill modules transfer to real robots without task-specific reward engineering?
• RQ4 How well can the same skill module support tasks like controllable walking and ball dribbling on different morphologies?

Key findings

• The skill modules enable faithful zero-shot imitation of MoCap clips on both ANYmal and OP3 in simulation and hardware for key motion primitives.
• The same skill module can be reused to achieve controllable walking across directions and velocities with high tracking accuracy in simulation and hardware.
• The modules can perform ball dribbling with high accuracy in both simulation and hardware, demonstrating goal-directed limb interactions.
• Domain and dynamics randomization plus an AR(1) prior stability regularization support robust sim-to-real transfer and natural motion styles.
• Regularization strength during imitation preserves style and enables better reuse; a schedule balancing imitation and reuse improves overall performance.