[Paper Review] Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos

Haoyu Xiong, Quanzhou Li+4 more|arXiv (Cornell University)|Jan 18, 2021

Multimodal Machine Learning Applications4 citations

TL;DR

This paper proposes Learning by Watching (LbW), a framework that enables robots to learn manipulation skills by imitating human videos through unsupervised human-to-robot image translation and keypoint detection. By generating semantically meaningful keypoint-based representations from translated robot videos, LbW achieves state-of-the-art performance on five manipulation tasks with only a single human demonstration.

ABSTRACT

Learning from visual data opens the potential to accrue a large range of manipulation behaviors by leveraging human demonstrations without specifying each of them mathematically, but rather through natural task specification. In this paper, we present Learning by Watching (LbW), an algorithmic framework for policy learning through imitation from a single video specifying the task. The key insights of our method are two-fold. First, since the human arms may not have the same morphology as robot arms, our framework learns unsupervised human to robot translation to overcome the morphology mismatch issue. Second, to capture the details in salient regions that are crucial for learning state representations, our model performs unsupervised keypoint detection on the translated robot videos. The detected keypoints form a structured representation that contains semantically meaningful information and can be used directly for computing reward and policy learning. We evaluate the effectiveness of our LbW framework on five robot manipulation tasks, including reaching, pushing, sliding, coffee making, and drawer closing. Extensive experimental evaluations demonstrate that our method performs favorably against the state-of-the-art approaches.

Motivation & Objective

• To enable physical imitation of manipulation skills from a single human video without requiring expert action annotations.
• To address the morphology mismatch between human and robot arms by learning unsupervised human-to-robot image translation.
• To improve state representation learning by detecting semantically meaningful keypoints in translated robot video frames.
• To develop a reward function based on keypoint trajectories that enables effective policy learning via reinforcement learning.
• To demonstrate generalization and robustness across diverse manipulation tasks using only one human demonstration.

Proposed method

• Leverages an unsupervised image-to-image translation model (e.g., CycleGAN) to convert human demonstration videos into robot-domain-like videos, minimizing domain shift.
• Performs unsupervised keypoint detection on the translated robot video frames to extract structured, semantically meaningful representations.
• Uses the detected keypoints as state representations for downstream policy learning, replacing conventional image-based features.
• Designs a reward function based on the L2 distance between the agent’s keypoint trajectory and the translated demonstration’s keypoint trajectory.
• Trains the policy using SAC+AE, a sample-efficient off-policy reinforcement learning algorithm, with the keypoint-based reward signal.
• Decouples perception and policy training: perception is trained on unpaired human and robot videos, while policy is trained on a single human video.

Figure 1 : LbW. Given a single human video, our LbW framework learns human to robot translation followed by unsupervised keypoint detection. The resulting keypoint-based representations are semantically meaningful and can be used to guide the robot to learn manipulation skills through physical imita

Experimental results

Research questions

• RQ1 Can a robot learn complex manipulation skills from a single human video without access to expert action sequences?
• RQ2 How can unsupervised image translation and keypoint detection jointly reduce the domain gap between human and robot visual domains?
• RQ3 Can keypoint-based state representations outperform standard image-based features in imitation learning from videos?
• RQ4 Does the proposed framework generalize across diverse manipulation tasks with minimal supervision?
• RQ5 What impact do visual artifacts in image-to-image translation have on downstream policy performance, and how can they be mitigated?

Key findings

• LbW achieves a 100% success rate on the reaching and pushing tasks, outperforming both AVID-m and classifier-reward baselines.
• On the more complex sliding, coffee making, and drawer closing tasks, LbW achieves higher success rates than AVID-m and the classifier-reward method, demonstrating superior generalization.
• Visual analysis confirms that AVID-m’s image translation produces significant artifacts (e.g., missing objects, mismatched poses), degrading policy performance.
• The keypoint-based representation enables more robust and semantically meaningful state encoding than direct feature extraction from translated images.
• LbW achieves state-of-the-art performance using only one human demonstration, while AVID-m uses 15 and the classifier-reward method uses 35 expert videos.
• The framework shows strong performance across two distinct simulation environments: OpenAI Gym and Meta-World, indicating broad applicability.

Figure 2 : Overview of the proposed LbW. Our LbW framework is composed of three main components: an image-to-image translation network $T$ , a keypoint detector $\Psi$ , and a policy network $\pi$ . The image-to-image translation network translates the input human demonstration video frame by frame