[Paper Review] Active Perception and Representation for Robotic Manipulation

Youssef Zaky, Mandakini Paruthi+2 more|arXiv (Cornell University)|Mar 15, 2020

Robot Manipulation and Learning42 references4 citations

TL;DR

This paper proposes an active perception and representation (APR) framework that integrates eye and hand control in robotic manipulation using a dual-policy approach: a fixation policy to actively control a camera for optimal viewpoint selection and a grasping policy for 6-DoF manipulation. By combining log-polar image encoding, self-supervised representation learning via Generative Query Networks (GQN), and joint policy training from sparse rewards, the model achieves 8% higher grasping success and at least four times greater sample efficiency than passive, fixed-camera baselines in simulated tasks.

ABSTRACT

The vast majority of visual animals actively control their eyes, heads, and/or bodies to direct their gaze toward different parts of their environment. In contrast, recent applications of reinforcement learning in robotic manipulation employ cameras as passive sensors. These are carefully placed to view a scene from a fixed pose. Active perception allows animals to gather the most relevant information about the world and focus their computational resources where needed. It also enables them to view objects from different distances and viewpoints, providing a rich visual experience from which to learn abstract representations of the environment. Inspired by the primate visual-motor system, we present a framework that leverages the benefits of active perception to accomplish manipulation tasks. Our agent uses viewpoint changes to localize objects, to learn state representations in a self-supervised manner, and to perform goal-directed actions. We apply our model to a simulated grasping task with a 6-DoF action space. Compared to its passive, fixed-camera counterpart, the active model achieves 8% better performance in targeted grasping. Compared to vanilla deep Q-learning algorithms, our model is at least four times more sample-efficient, highlighting the benefits of both active perception and representation learning.

Motivation & Objective

• To address the inefficiency of passive vision in deep reinforcement learning for robotic manipulation by incorporating active perception inspired by primate visual systems.
• To enable efficient, self-supervised learning of scene representations from raw visual observations using multiple viewpoints.
• To jointly train fixation and grasping policies using a single reward signal, improving sample efficiency and task performance.
• To extend sample-efficient 6-DoF grasping beyond 4-DoF top-down control, enabling more realistic object manipulation.
• To demonstrate that active viewpoint control and space-variant visual sampling enhance representation quality and downstream policy learning.

Proposed method

• A second 6-DoF manipulator controls a wrist-mounted camera to actively shift the viewpoint and foveate on task-relevant objects.
• Images are processed using a log-polar transform to emphasize the central region, mimicking the primate foveal visual system.
• A multimodal encoder combines visual and proprioceptive inputs from two viewpoints to form a combined scene representation.
• Generative Query Networks (GQN) are used to learn a disentangled, low-dimensional scene representation from multi-view observations.
• A joint policy network learns both fixation and grasping behaviors using a shared reward function, enabling end-to-end training from sparse rewards.
• The model uses a single reward signal to train both policies, with fixation guiding the grasping policy toward relevant objects.

Experimental results

Research questions

• RQ1 Can active viewpoint control improve sample efficiency and performance in vision-based robotic manipulation?
• RQ2 How does self-supervised multi-view representation learning from active perception enhance policy learning compared to passive observation?
• RQ3 Can a unified policy framework jointly learn fixation and grasping behaviors using a single reward signal?
• RQ4 To what extent does space-variant visual sampling (via log-polar transform) improve representation quality and task performance?
• RQ5 How does active perception compare to fixed-camera baselines in 6-DoF grasping tasks?

Key findings

• The active perception model achieved 85% success rate on test objects in under 70,000 grasp attempts, outperforming the passive fixed-camera baseline by 8%.
• The APR model demonstrated at least fourfold improvement in sample efficiency compared to vanilla deep Q-learning algorithms, highlighting the benefit of active perception and representation learning.
• Representation quality improved over training, as shown by increasingly sharp GQN-generated images of the central object, correlating with performance gains.
• The model successfully learned 6-DoF grasping policies that adaptively orient the gripper based on object geometry and scene context.
• The fixation policy learned to direct attention toward task-relevant objects, implicitly defining goals for the grasping policy.
• The combination of active perception and self-supervised representation learning was the primary driver of sample efficiency gains, outperforming passive models even when both used representation learning.