[Paper Review] Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning

Frederik Ebert, Sudeep Dasari+3 more|arXiv (Cornell University)|Oct 6, 2018

Domain Adaptation and Few-Shot Learning41 citations

TL;DR

Proposes a closed-loop visual MPC framework that uses self-supervised image registration to keep track of goals and enable retrying, achieving long-horizon manipulation from raw images with autonomously collected data.

ABSTRACT

Prediction is an appealing objective for self-supervised learning of behavioral skills, particularly for autonomous robots. However, effectively utilizing predictive models for control, especially with raw image inputs, poses a number of major challenges. How should the predictions be used? What happens when they are inaccurate? In this paper, we tackle these questions by proposing a method for learning robotic skills from raw image observations, using only autonomously collected experience. We show that even an imperfect model can complete complex tasks if it can continuously retry, but this requires the model to not lose track of the objective (e.g., the object of interest). To enable a robot to continuously retry a task, we devise a self-supervised algorithm for learning image registration, which can keep track of objects of interest for the duration of the trial. We demonstrate that this idea can be combined with a video-prediction based controller to enable complex behaviors to be learned from scratch using only raw visual inputs, including grasping, repositioning objects, and non-prehensile manipulation. Our real-world experiments demonstrate that a model trained with 160 robot hours of autonomously collected, unlabeled data is able to successfully perform complex manipulation tasks with a wide range of objects not seen during training.

Motivation & Objective

• Enable long-horizon robotic manipulation from raw RGB observations using self-supervised learning.
• Allow a robot to continuously retry tasks by retaining alignment to the goal despite prediction errors.
• Learn both prehensile and non-prehensile manipulation from autonomously collected data without human supervision.
• Provide a grounded planning cost based on image-to-image registration to guide video-prediction based control.

Proposed method

• Train a video-prediction model to forecast future observations conditioned on action sequences.
• Develop a self-supervised image registration model that aligns current observations to the start and goal images.
• Define a planning cost as a weighted pixel-distance between registered current and goal positions, updated via registration quality.
• Use model-predictive control with re-planning at each real-world step to enable retrying.
• Extend to multi-view (stereo) setups to define 3D goals and combine costs across views.
• Incorporate a simple grasping reflex to enable combined prehensile and non-prehensile manipulation.

Experimental results

Research questions

• RQ1 Can self-supervised predictive models of raw visual observations be used for temporally extended robotic manipulation tasks?
• RQ2 Does continuous retrying with a learned registration-based cost improve success on long-horizon object relocation tasks?
• RQ3 What is the impact of using image registration as planning cost versus OpenCV trackers or predictor propagation?
• RQ4 Can the approach scale to multiple cameras and enable 3D goal specification?

Key findings

• A registration-based planning cost substantially improves performance on temporally extended manipulation tasks compared to predictor-propagation and OpenCV trackers.
• Closed-loop retrying enabled by the registration module leads to higher success rates on long-distance pushing tasks.
• The method enables both non-prehensile and combined prehensile/non-prehensile manipulation from purely autonomous data without supervision.
• On long-distance pushing benchmarks with unseen objects, the registration-based visual MPC outperformed OpenCV-based tracking and predictor propagation in success rate.
• Combining multiple camera views allows defining 3D goals and solving tasks that are ambiguous from a single view.
• A simple grasping reflex during data collection allows learning grasping skills alongside pushing, enabling more versatile manipulation.