[Paper Review] A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms

Oliver Kroemer, Scott Niekum+1 more|arXiv (Cornell University)|Jul 6, 2019

Robot Manipulation and Learning469 references97 citations

TL;DR

This paper surveys how machine learning is applied to robot manipulation, formalizes a structured manipulation learning problem, and outlines core representations, learning challenges, and task-family transfer.

ABSTRACT

A key challenge in intelligent robotics is creating robots that are capable of directly interacting with the world around them to achieve their goals. The last decade has seen substantial growth in research on the problem of robot manipulation, which aims to exploit the increasing availability of affordable robot arms and grippers to create robots capable of directly interacting with the world to achieve their goals. Learning will be central to such autonomous systems, as the real world contains too much variation for a robot to expect to have an accurate model of its environment, the objects in it, or the skills required to manipulate them, in advance. We aim to survey a representative subset of that research which uses machine learning for manipulation. We describe a formalization of the robot manipulation learning problem that synthesizes existing research into a single coherent framework and highlight the many remaining research opportunities and challenges.

Motivation & Objective

• Summarize a formal framework for robot manipulation learning that unifies existing work.
• Highlight the key structural properties of manipulation tasks (physics, underactuation, hierarchy) to enable learning.
• Discuss representations and perception (object-centric, passive/interactive) to support learning across task families.
• Describe learning approaches for state space discovery, transition models, motor policies, skill modeling, and hierarchical abstractions.
• Identify open challenges and opportunities for transfer across tasks and open-world settings.

Proposed method

• Formalize manipulation learning as a structured collection of MDPs (task family) with shared action space and task-specific states, rewards, and context.
• Adopt object-centric state and context factorization to enable object-level generalization across tasks.
• Describe hybrid, piecewise-continuous dynamics (modes) to capture underactuated manipulation and mode switches.
• Incorporate options (skills) and hierarchical representations to enable reusable motor skills across tasks.
• Differentiate passive and interactive perception and discuss active/self-supervised learning for improving object properties and state representations.
• Propose learning across a task family via context-aware policies that transfer across tasks with varying object properties and environmental configurations.

Experimental results

Research questions

• RQ1 How can manipulation tasks be formalized to support learning across a family of related tasks?
• RQ2 What representations (object-centric, hierarchical) best support generalization across tasks and objects?
• RQ3 How can state, transition, and reward models be learned and transferred across a task family?
• RQ4 What role do interactive perception and active learning play in efficient manipulation learning?
• RQ5 How can skills be defined, transferred, and composed to solve new tasks within a task family?

Key findings

• Manipulation tasks are naturally modeled as structured MDPs with object-centric state spaces enabling transfer across task instances.
• Underactuation and mode switches create hybrid dynamics that learning algorithms must handle, often via hierarchical or skill-based approaches.
• Object-centric representations (point, part, object levels) and interactive perception improve generalization across tasks and objects.
• Skills (options) and hierarchical decompositions enable reusable motor policies and modular learning across task families.
• Interactive perception and active learning can reduce uncertainty and enable self-supervised grounding of passive perception for improved modeling and control.
• Transfer across a task family relies on context vectors and object-level structure to generalize policies and models to new tasks.