[Paper Review] Self-Supervised Visual Planning with Temporal Skip Connections

Frederik Ebert, Chelsea Finn+2 more|arXiv (Cornell University)|Oct 15, 2017

Robot Manipulation and Learning20 references113 citations

TL;DR

The paper introduces an occlusion-aware video prediction model (SNA) with temporal skip connections and a distance-based planning objective for visual MPC, enabling planning through occlusions and handling multiple objects using a hybrid continuous-discrete action space.

ABSTRACT

In order to autonomously learn wide repertoires of complex skills, robots must be able to learn from their own autonomously collected data, without human supervision. One learning signal that is always available for autonomously collected data is prediction: if a robot can learn to predict the future, it can use this predictive model to take actions to produce desired outcomes, such as moving an object to a particular location. However, in complex open-world scenarios, designing a representation for prediction is difficult. In this work, we instead aim to enable self-supervised robotic learning through direct video prediction: instead of attempting to design a good representation, we directly predict what the robot will see next, and then use this model to achieve desired goals. A key challenge in video prediction for robotic manipulation is handling complex spatial arrangements such as occlusions. To that end, we introduce a video prediction model that can keep track of objects through occlusion by incorporating temporal skip-connections. Together with a novel planning criterion and action space formulation, we demonstrate that this model substantially outperforms prior work on video prediction-based control. Our results show manipulation of objects not seen during training, handling multiple objects, and pushing objects around obstructions. These results represent a significant advance in the range and complexity of skills that can be performed entirely with self-supervised robotic learning.

Motivation & Objective

• Motivate self-supervised robot learning from autonomously collected data via video prediction.
• Develop an occlusion-aware predictive model that preserves object permanence across occlusions.
• Improve planning for vision-based control by using a smooth distance-based cost over pixel locations.
• Enable planning with both discrete and continuous actions within a model-predictive control framework.

Proposed method

• Propose a Skip Connection Neural Advection (SNA) model that extends DNA with temporal skip connections to maintain object permanence through occlusions.
• Predict next frames by compositing multiple transformed past images with learned masks, enabling occlusion handling by copying from history.
• Use a distance-based planning objective that minimizes the expected Euclidean distance between predicted pixel positions and a goal, across a horizon T.
• Adopt a sampling-based model-predictive control (CEM) with a hybrid action space that combines continuous end-effector motions and discrete lift actions.
• Represent actions as a vector that includes horizontal motion and a discrete lift level, with rounding to nearest discrete step for optimization.
• Train the video prediction model from randomly collected pushing trajectories without external supervision.

Experimental results

Research questions

• RQ1 Can an occlusion-aware video prediction model track designated pixels through occlusions during manipulation?
• RQ2 Does a distance-based planning objective improve long-horizon visual MPC performance under occlusions?
• RQ3 Can a hybrid action space (continuous plus discrete lift) be effectively integrated into sampling-based MPC for tabletop manipulation?
• RQ4 How does the proposed SNA model compare to prior DNA-based approaches in occlusion-heavy tasks and unseen objects?

Key findings

• The SNA model substantially improves planning performance in occlusion-heavy tasks compared to prior DNA-based methods.
• Using the expected-distance cost over predicted pixel positions leads to better long-horizon planning than previous probability-based costs.
• The hybrid action space enables lifting the end-effector to move over obstacles, producing more natural and shorter trajectories.
• SNA maintains prediction quality for occluded objects, enabling successful planning with unseen objects and multiple objects.
• Experiments show pushing tasks with occlusions and multi-object setups where self-supervised video prediction guides control.
• SNA with the new planning cost achieves competitive or superior results to prior methods on seen and unseen objects.