[Paper Review] Time-Contrastive Networks: Self-Supervised Learning from Video

Pierre Sermanet, Corey Lynch+5 more|arXiv (Cornell University)|Apr 23, 2017

Human Pose and Action Recognition48 references53 citations

TL;DR

The paper introduces Time-Contrastive Networks (TCN), a self-supervised, multi-view representation learning method from unlabeled videos, enabling third-person imitation and RL-based robot control using only visual input.

ABSTRACT

We propose a self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses. Imitation of human behavior requires a viewpoint-invariant representation that captures the relationships between end-effectors (hands or robot grippers) and the environment, object attributes, and body pose. We train our representations using a metric learning loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. In other words, the model simultaneously learns to recognize what is common between different-looking images, and what is different between similar-looking images. This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm. While representations are learned from an unlabeled collection of task-related videos, robot behaviors such as pouring are learned by watching a single 3rd-person demonstration by a human. Reward functions obtained by following the human demonstrations under the learned representation enable efficient reinforcement learning that is practical for real-world robotic systems. Video results, open-source code and dataset are available at https://sermanet.github.io/imitate

Motivation & Objective

• Learn viewpoint-invariant, disentangled representations of object interactions and pose from unlabeled multi-view videos.
• Enable imitation of human behavior from third-person video without explicit pose labels or correspondences.
• Provide a reward signal for reinforcement learning using TCN embeddings learned from video data.
• Demonstrate pouring and dish-rack manipulation tasks in simulation and on real robots using TCN-based guidance.

Proposed method

• Train embedding f(x) using a triplet loss on co-occurring frames from different viewpoints (anchor, positive) vs. a temporally nearby negative.
• Use multi-view data to ground and disambiguate visual changes, achieving viewpoint, occlusion, lighting, and background invariances.
• Optionally use a single-view TC loss with a defined positive window when multi-view data is unavailable.
• Leverage a 32-dimensional TCN embedding to form a reward function for reinforcement learning via a squared distance term plus a Huber-style term.
• Integrate TCN features into PILQR-based policy optimization to learn manipulation tasks from video demonstrations.
• Apply direct pose imitation by self-regression with a shared TCN embedding trained on human and robot motions.

Experimental results

Research questions

• RQ1 Can Time-Contrastive Networks learn representations that disentangle pose and object interactions while being invariant to viewpoint and appearance?
• RQ2 Can the learned TCN embedding provide a robust reward signal for RL to acquire complex manipulation skills from third-person demonstrations?
• RQ3 Is imitation from third-person video feasible without explicit pose or correspondence labels?
• RQ4 How do multi-view versus single-view training signals affect representation quality and robotic learning outcomes?
• RQ5 Can TCN support real-time, continuous imitation of human poses without pose labels?

Key findings

• Multi-view TCNs outperform baselines in both alignment and attribute classification for pouring tasks.
• mvTCN enables efficient real-world pouring and dish-rack manipulation, with pouring performance converging after about 10 iterations on a real robot.
• Single-view TCNs and shuffle-and-learn baselines underperform relative to mvTCN, despite identical data; multi-view signals accelerate learning.
• TCN-based rewards enable PILQR-based reinforcement learning to learn pouring with a real robot and a simulated dish rack task, outperforming other representations.
• Direct pose imitation via self-regression with a shared TCN embedding enables end-to-end imitation without joint-level pose labels, and can be augmented with limited human supervision.
• The approach demonstrates strong qualitative results, including robust imitation from third-person videos and rapid task acquisition.