[Paper Review] Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation

Tianhao Zhang, Zoe McCarthy+5 more|arXiv (Cornell University)|Oct 12, 2017

Robot Manipulation and Learning50 references54 citations

TL;DR

The authors build a consumer-VR teleoperation system to collect demonstrations and train deep visuomotor policies via behavioral cloning, achieving high success across ten real-world manipulation tasks with less than 30 minutes of data per task.

ABSTRACT

Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills.

Motivation & Objective

• Demonstrate an inexpensive VR teleoperation setup for collecting high-quality robotic manipulation demonstrations.
• Learn deep visuomotor policies that map from pixels to actions using behavioral cloning.
• Show that a single neural architecture can handle multiple manipulation tasks with limited data.
• Analyze sample efficiency and the impact of auxiliary self-supervision losses on learning.
• Explore generalization to unseen initial states and task variations.

Proposed method

• Use Vive VR headset and motion-tracked controllers to teleoperate a PR2 robot with shared observation and action spaces.
• Capture RGB-D images and end-effector history as inputs; output gripper command, angular and linear velocities for the right arm.
• Adopt a neural network with three modules (vision, auxiliary, control) and a spatial soft-argmax for feature extraction from images.
• Train with a behavioral cloning loss comprising L2, L1, directional alignment loss, and a gripper open/close loss plus auxiliary losses.
• Incorporate self-supervised auxiliary tasks (predicting current and final gripper poses, and other task-relevant signals) to boost data efficiency.
• Optimize using stochastic gradient descent/ADAM with a fixed hyperparameter set across tasks.

Experimental results

Research questions

• RQ1 Can inexpensive VR teleoperation collect demonstrations suitable for learning visuomotor policies from pixels?
• RQ2 What is the data efficiency (demonstration time) needed to learn successful policies for diverse manipulation tasks?
• RQ3 Do auxiliary self-supervision losses improve data efficiency on real-world robotic manipulation?
• RQ4 How well do policies generalize to unseen initial states and task variations?

Key findings

• VR demonstrations enabled high-performance deep visuomotor policies for a range of real-world tasks.
• Most tasks achieved high success with under 30 minutes of demonstrations per task.
• Demonstrations collected via VR provided sufficient natural variation for learning.
• Auxiliary prediction losses improved data efficiency in real-world manipulation.
• Policies showed generalization to unseen initial states and extrapolated scenarios.
• The approach could handle long-horizon, multi-stage tasks and complex contact-rich manipulations.