[Paper Review] One-Shot Visual Imitation Learning via Meta-Learning

Chelsea Finn, Tianhe Yu+3 more|arXiv (Cornell University)|Sep 14, 2017

Human Pose and Action Recognition27 references267 citations

TL;DR

The paper combines gradient-based meta-learning with imitation learning to enable a robot to acquire new skills from a single visual demonstration, scalable to raw pixel inputs and even video-only demonstrations.

ABSTRACT

In order for a robot to be a generalist that can perform a wide range of jobs, it must be able to acquire a wide variety of skills quickly and efficiently in complex unstructured environments. High-capacity models such as deep neural networks can enable a robot to represent complex skills, but learning each skill from scratch then becomes infeasible. In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration. Unlike prior methods for one-shot imitation, our method can scale to raw pixel inputs and requires data from significantly fewer prior tasks for effective learning of new skills. Our experiments on both simulated and real robot platforms demonstrate the ability to learn new tasks, end-to-end, from a single visual demonstration.

Motivation & Objective

• Motivate the need for generalist robots that learn many skills quickly.
• Improve data efficiency by reusing experience across tasks.
• Enable end-to-end learning from raw visual inputs for fast adaptation.
• Develop a meta-learning framework that enables one-shot imitation from a single demonstration.
• Demonstrate real-world applicability on simulated and physical robots.

Proposed method

• Extend model-agnostic meta-learning (MAML) to imitation learning from visual inputs.
• Train a policy that adapts to a new task with one gradient update using a demonstration.
• Introduce a two-head architecture where the final layer is effectively a meta-learned inner objective.
• Incorporate a bias transformation to enhance gradient-based adaptation.
• Meta-train with pairs of demonstrations per task to simulate training/validation split for adaptation.
• Optionally learn a loss function (through the two-head setup) to enable adaptation without expert actions.

Experimental results

Research questions

• RQ1 Can one-shot imitation learning from raw pixel inputs adapt to unseen tasks from a single visual demonstration?
• RQ2 How does meta-imitation learning (MIL) compare to prior one-shot imitation methods across dataset sizes and input modalities (vision vs. non-vision)?
• RQ3 Can the method learn from demonstrations that do not include expert actions (video-only demonstrations)?
• RQ4 Does MIL scale to real-world robotic tasks with real images and objects?

Key findings

• MIL outperforms prior one-shot imitation methods on vision-based tasks that require adapting to new objects and goals.
• In simulated pushing with held-out objects, MIL achieves up to 85.81% one-shot success when provided with video+state+action inputs, outperforming baselines.
• MIL achieves 68.33% one-shot success with video-only demonstrations in pushing tasks.
• In real-world placing with a PR2 robot, MIL reaches 90% one-shot success on placing held items into the correct container using a single demonstration.
• MIL with video-only demonstrations remains feasible (68.33% in placing) but benefits from additional data or modalities.
• Two-head and bias-transformation variants improve stability and performance of gradient-based adaptation.