[Paper Review] Reaching, Grasping and Re-grasping: Learning Multimode Grasping Skills

Wenbin Hu, Chuanyu Yang+2 more|arXiv (Cornell University)|Feb 11, 2020

Robot Manipulation and Learning31 references4 citations

TL;DR

This paper proposes a unified deep reinforcement learning policy for robotic grasping that seamlessly integrates reaching, grasping, and re-grasping in dynamic environments. By using a task-oriented reward function and training with failure-inducing initial states, the policy achieves robust, collision-free re-grasping after failures, with high success rates on static and moving targets in simulation, demonstrating real-time adaptation and strong generalization under partial, noisy visual observations.

ABSTRACT

The ability to adapt to uncertainties, recover from failures, and coordinate between hand and fingers are essential sensorimotor skills for fully autonomous robotic grasping. In this paper, we aim to study a unified feedback control policy for generating the finger actions and the motion of hand to accomplish seamlessly coordinated tasks of reaching, grasping and re-grasping. We proposed a set of quantified metrics for task-orientated rewards to guide the policy exploration, and we analyzed and demonstrated the effectiveness of each reward term. To acquire a robust re-grasping motion, we deployed different initial states in training to experience failures that the robot would encounter during grasping due to inaccurate perception or disturbances. The performance of learned policy is evaluated on three different tasks: grasping a static target, grasping a dynamic target, and re-grasping. The quality of learned grasping policy was evaluated based on success rates in different scenarios and the recovery time from failures. The results indicate that the learned policy is able to achieve stable grasps of a static or moving object. Moreover, the policy can adapt to new environmental changes on the fly and execute collision-free re-grasp after a failed attempt within a short recovery time even in difficult configurations.

Motivation & Objective

• To develop a unified feedback control policy that coordinates reaching, grasping, and re-grasping in a single, reactive framework.
• To improve robustness in real-world scenarios by training the policy to recover from failures using induced failure states during training.
• To enable real-time adaptation to sudden environmental changes and partial/noisy visual observations.
• To reduce reliance on pre-collected data or manual controller design by learning from scratch via trial-and-error in simulation.
• To facilitate sim-to-real transfer by incorporating realistic constraints and sensor noise into the training process.

Proposed method

• A model-free deep reinforcement learning framework is used to train the policy end-to-end from raw observations, including object key points and in-hand contact forces.
• A multi-component reward function is designed with terms for distance to target, finger vector alignment, topology consistency, contact force, object velocity, and collision avoidance.
• Failure-inducing initial states are explicitly introduced during training to expose the agent to re-grasping scenarios and improve recovery behavior.
• Partial and noisy visual observations are simulated by using only half the object's key points and adding Gaussian noise (σ = 0.02m) to replicate real-world sensor limitations.
• Motion constraints are enforced via penalties on peak finger torque (≤2 N·m) and hand velocity (≤1 m/s) to ensure safety and hardware feasibility.
• The policy is trained and evaluated in a PyBullet-based simulation environment using a three-fingered robot hand and a Franka arm kinematic chain.

Experimental results

Research questions

• RQ1 Can a single deep reinforcement learning policy effectively coordinate reaching, grasping, and re-grasping in a unified manner?
• RQ2 How do individual components of the reward function contribute to the policy’s success and robustness?
• RQ3 To what extent can the policy recover from grasping failures and re-grasp efficiently in complex configurations?
• RQ4 How well does the policy generalize to unseen objects and partial/noisy visual inputs?
• RQ5 Can the policy be transferred to real hardware, given realistic training constraints?

Key findings

• The policy achieved 100% success rate in grasping static targets and 95% success rate in grasping dynamic targets under controlled conditions.
• The policy successfully executed collision-free re-grasping after failure within an average recovery time of less than 0.5 seconds.
• Ablation studies showed that removing the distance-to-finger-tips reward term (r_distTips) led to complete failure in all tasks, highlighting its critical role.
• The policy generalized well to unseen objects, including a cylinder, can, mustard bottle, and wood block, demonstrating robustness to shape and size variation.
• Peak finger joint torques remained below 2 N·m and hand velocities below 1 m/s across all tasks, confirming compliance with hardware safety limits.
• The policy remained robust under partial and noisy visual observations, maintaining high performance even when only 50% of object key points were used and noise was added.