[Paper Review] Defining the problem of Observation Learning

Leo Pauly|arXiv (Cornell University)|Aug 24, 2018

Robot Manipulation and Learning11 references3 citations

TL;DR

This paper defines observation learning as a distinct problem in robotics, where agents learn tasks by observing demonstrators in third-person (allocentric) views without explicit programming. It proposes a two-stage learning framework: first mapping raw video observations to a viewpoint-invariant feature space, then mapping those features to robotic control variables, enabling human-like skill acquisition through visual demonstration.

ABSTRACT

This article defines and formulates the problem of observation learning in robotic systems.

Motivation & Objective

• To formally define observation learning as a distinct problem from imitation learning, emphasizing third-person (allocentric) observation.
• To address the challenge of enabling robots to learn new tasks by observing human demonstrations without explicit programming.
• To develop a learning framework that maps raw visual inputs from demonstrations to robotic control policies, mimicking human learning.
• To differentiate observation learning from imitation learning by focusing on viewpoint-invariant feature learning from unstructured video inputs.
• To lay the foundation for scalable, human-like skill acquisition in robotics through visual observation alone.

Proposed method

• Formulate observation learning as a mapping M: X → C, where X is a video sequence of a human demonstrator and C is the robotic manipulator's control variables (joint angles, speed, torque).
• Decompose the mapping into two stages: M₁: X → F (mapping raw video to a viewpoint-invariant feature space F), and M₂: F → C (mapping features to robot control outputs).
• Ensure M₁ produces identical outputs for the same action viewed from different angles, making F invariant to observer viewpoint.
• Use self-supervision to learn the mapping between the robot’s own 3rd-person view and its joint configurations.
• Use human-supervision to learn the mapping between human demonstrator’s 3rd-person observations and the robot’s joint configurations.
• Combine human-supervision, self-supervision, and observation learning in an iterative training process to learn the X → C mapping.

Experimental results

Research questions

• RQ1 How can observation learning be formally distinguished from imitation learning, particularly in terms of perspective (allocentric vs. egocentric)?
• RQ2 What is a suitable mathematical formulation for mapping raw video demonstrations to robotic control policies?
• RQ3 How can viewpoint-invariant feature representations be learned from unstructured video inputs to enable robust task generalization?
• RQ4 What role do human-supervision and self-supervision play in learning the mapping between human actions and robotic control?
• RQ5 What are the limitations of current assisted observation learning methods that rely on hand-engineered features?

Key findings

• Observation learning is a distinct problem from imitation learning due to the use of allocentric (third-person) observation, requiring separate modeling and tools.
• The proposed two-stage mapping—M₁ from video to viewpoint-invariant features, M₂ from features to control—enables robust learning from diverse viewing angles.
• Self-supervision and human-supervision are essential for learning the correspondence between visual observations and robotic joint configurations.
• The combined use of human-supervision, self-supervision, and observation learning improves performance in real-world, unconstrained environments.
• Assisted observation learning, while simpler, is limited by the need for hand-engineered features and exact imitation of robot movements, which is impractical to collect.
• The field remains in its infancy, especially in implicit observation learning, with significant potential for future development in human-like skill acquisition.