[Paper Review] One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning

Tianhe Yu, Chelsea Finn+5 more|arXiv (Cornell University)|Feb 5, 2018

Robot Manipulation and LearningEngineering56 references111 citations

TL;DR

The paper presents a domain-adaptive meta-learning approach that enables a robot to imitate a new task from a single video of a human, using prior human and robot demonstrations across tasks.

ABSTRACT

Humans and animals are capable of learning a new behavior by observing others perform the skill just once. We consider the problem of allowing a robot to do the same -- learning from a raw video pixels of a human, even when there is substantial domain shift in the perspective, environment, and embodiment between the robot and the observed human. Prior approaches to this problem have hand-specified how human and robot actions correspond and often relied on explicit human pose detection systems. In this work, we present an approach for one-shot learning from a video of a human by using human and robot demonstration data from a variety of previous tasks to build up prior knowledge through meta-learning. Then, combining this prior knowledge and only a single video demonstration from a human, the robot can perform the task that the human demonstrated. We show experiments on both a PR2 arm and a Sawyer arm, demonstrating that after meta-learning, the robot can learn to place, push, and pick-and-place new objects using just one video of a human performing the manipulation.

Motivation & Objective

• Develop a method for one-shot imitation from raw human videos despite domain shifts between human and robot.
• Leverage meta-learning with prior human and robot demonstrations across tasks to acquire a transferable policy prior.
• Enable rapid adaptation to new tasks with only one human demonstration and no robot data for the new task.

Proposed method

• Extend MAML to handle domain shift and learning from human videos without human actions.
• Meta-train with paired human and robot demonstrations across tasks to learn initialization and a learned adaptation loss L_ψ.
• Learn a temporal adaptation objective L_ψ using 1D temporal convolutions to process video tricks and guide gradient updates.
• Use a policy that maps RGB inputs to action distributions, with perception via spatial soft-argmax and a mixture density for actions.
• During meta-testing, adapt from the learned initialization using one human demonstration to produce robot actions.
• Provide a probabilistic interpretation that frames adaptation as MAP inference with a learned energy term.

Experimental results

Research questions

• RQ1 Can the proposed domain-adaptive meta-learning enable a robot to imitate new tasks from a single human video across object and scene variations?
• RQ2 Does the approach generalize to different perspectives, backgrounds, and demonstrators not seen during training?
• RQ3 How does the temporal adaptation objective compare to per-timestep losses in leveraging video information?
• RQ4 Is the method transferable across different robot platforms and demonstration types (kinesthetic/teleoperation)?

Key findings

• Robust one-shot imitation across new objects and tasks after meta-training with human and robot data.
• Temporal adaptation objective significantly outperforms linear per-timestep loss, enabling better learning from video demonstrations.
• On PR2, DAML temporal loss achieves high success rates for placing (93.8%), pushing (88.9%), and pick-and-place (80.0%).
• Baseline DA-LSTM and contextual policies struggle with limited demonstrations, highlighting data efficiency of the proposed temporal loss.
• The method remains effective under large domain shifts, including different camera viewpoints and backgrounds, though background variation can reduce performance.