[Paper Review] MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

Chen Wang, Linxi Fan+6 more|arXiv (Cornell University)|Feb 24, 2023

Robot Manipulation and Learning24 citations

TL;DR

MimicPlay learns a 3D-aware latent planner from cheap human play data to guide a low-level visuomotor controller trained on a small set of teleoperation demonstrations, enabling efficient, robust long-horizon manipulation across 14 real-world tasks.

ABSTRACT

Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often requires an unattainable amount of demonstrations. To reduce the high data requirement, we resort to human play data - video sequences of people freely interacting with the environment using their hands. Even with different morphologies, we hypothesize that human play data contain rich and salient information about physical interactions that can readily facilitate robot policy learning. Motivated by this, we introduce a hierarchical learning framework named MimicPlay that learns latent plans from human play data to guide low-level visuomotor control trained on a small number of teleoperated demonstrations. With systematic evaluations of 14 long-horizon manipulation tasks in the real world, we show that MimicPlay outperforms state-of-the-art imitation learning methods in task success rate, generalization ability, and robustness to disturbances. Code and videos are available at https://mimic-play.github.io

Motivation & Objective

• Reduce data requirements for long-horizon imitation learning by leveraging cheap human play data to learn high-level plans.
• Decouple planning and control via a latent plan space to guide low-level visuomotor policies.
• Bridge human and robot embodiments with a 3D-aware latent planner and a small set of robot demonstrations.
• Demonstrate improved sample efficiency, generalization, and robustness across diverse real-world tasks.

Proposed method

• Learn a goal-conditioned latent planner from human play data that predicts future 3D human hand trajectories given a goal image.
• Model the trajectory distribution with an MLP-based Gaussian Mixture Model to capture multimodality.
• Minimize a KL divergence between human and robot vision encodings to reduce visual domain gap.
• Train a plan-guided multi-task low-level controller (transformer-based) that maps latent plans and sensor signals to actions using a small amount of teleoperation data.
• Prompt robot motion with one-shot videos (human or robot) to generate latent plans guiding the low-level controller.
• Use a two-stage training: (1) latent planner from human play data; (2) low-level controller trained with robot demonstrations conditioned on latent plans.

Experimental results

Research questions

• RQ1 Can cheap human play data learn a robust, multimodal latent plan space that guides long-horizon robot manipulation?
• RQ2 Does decomposing planning and control improve sample efficiency and generalization over end-to-end methods in long-horizon tasks?
• RQ3 How does a small amount of teleoperation data combined with latent plans perform across multiple tasks and unseen subgoal compositions?
• RQ4 Can prompting with human videos serve as effective goals for robot manipulation via the latent planner?

Key findings

• Latent plans learned from human play data significantly improve performance, achieving substantial gains in long-horizon tasks with minimal teleoperation data.
• Two-stage, hierarchical learning outperforms end-to-end methods in long-horizon settings.
• GMM is crucial for capturing multimodal human trajectories; removing GMM degrades performance and generalization.
• KL-based alignment between human and robot visual representations reduces domain gap and improves planning.
• More human play data improves generalization to unseen subgoal compositions and tasks, with prompts from human videos competitive with robot prompts.