[Paper Review] Deep Differentiable Grasp Planner for High-DOF Grippers

Min Liu, Zherong Pan+3 more|arXiv (Cornell University)|Feb 4, 2020

Robot Manipulation and Learning55 references4 citations

TL;DR

This paper proposes a differentiable grasp planner for high-DOF grippers that enables end-to-end training of deep neural networks to predict grasp poses using automatic differentiation. By formulating a generalized, differentiable grasp quality metric and collision constraints from surface meshes, the method achieves superior grasp success rates with minimal ground truth data, improving physical hardware success by 22% and Q₁ metric by 0.12 over prior methods.

ABSTRACT

We present an end-to-end algorithm for training deep neural networks to grasp novel objects. Our algorithm builds all the essential components of a grasping system using a forward-backward automatic differentiation approach, including the forward kinematics of the gripper, the collision between the gripper and the target object, and the metric for grasp poses. In particular, we show that a generalized Q1 grasp metric is defined and differentiable for inexact grasps generated by a neural network, and the derivatives of our generalized Q1 metric can be computed from a sensitivity analysis of the induced optimization problem. We show that the derivatives of the (self-)collision terms can be efficiently computed from a watertight triangle mesh of low-quality. Altogether, our algorithm allows for the computation of grasp poses for high-DOF grippers in an unsupervised mode with no ground truth data, or it improves the results in a supervised mode using a small dataset. Our new learning algorithm significantly simplifies the data preparation for learning-based grasping systems and leads to higher qualities of learned grasps on common 3D shape datasets [7, 49, 26, 25], achieving a 22% higher success rate on physical hardware and a 0.12 higher value on the Q1 grasp quality metric.

Motivation & Objective

• To simplify data preparation in learning-based grasping by eliminating reliance on large-scale ground truth grasp annotations.
• To enable end-to-end training of deep neural networks for high-DOF grippers using only multi-view depth images as input.
• To develop a differentiable grasp quality metric that remains valid even for inexact, non-contact grasps.
• To efficiently compute gradients for self-collision and object-collision constraints using low-quality triangle meshes.
• To improve grasp quality and success rate in both unsupervised and supervised learning regimes with minimal data.

Proposed method

• The method uses a neural network that takes multi-view depth images as input and directly predicts high-DOF gripper configurations via ResNet-50 and fully connected layers with view pooling.
• Forward kinematics transforms the predicted configuration into Euclidean space for physical execution and collision checking.
• A differentiable grasp quality metric, generalized as Q₁, is defined and optimized even when the gripper does not contact the object.
• The gradient of the Q₁ metric is computed via sensitivity analysis of the underlying optimization problem, enabling backpropagation through the grasp quality function.
• Self-collision and object-collision constraints are enforced using differentiable distance functions computed from watertight triangle meshes.
• The full system is trained using a loss function combining grasp quality maximization, collision avoidance, data consistency, and gripper-object proximity.

Experimental results

Research questions

• RQ1 Can a differentiable grasp quality metric be defined and optimized for inexact, non-contact grasps in high-DOF grippers?
• RQ2 How can collision constraints for self-collisions and object collisions be made differentiable using only surface mesh representations?
• RQ3 Can a deep neural network be trained end-to-end to predict high-DOF grasp poses without requiring large-scale ground truth grasp data?
• RQ4 To what extent can differentiable grasp planning improve grasp success rates and quality metrics in real-world hardware deployments?
• RQ5 Can the proposed method reduce data preparation complexity while maintaining or improving performance in learning-based grasping?

Key findings

• The proposed method achieves a 22% higher grasp success rate on physical hardware compared to baseline methods.
• The method improves the Q₁ grasp quality metric by 0.12 points, indicating higher grasp stability and robustness.
• The approach enables unsupervised training with no ground truth grasp data, significantly reducing data preparation overhead.
• The differentiable grasp metric and collision constraints allow for efficient gradient-based optimization, leading to locally optimal grasp poses.
• The method maintains high performance even with low-quality triangle meshes, enabling efficient collision computation.
• The architecture achieves state-of-the-art performance on standard 3D shape datasets, including ShapeNet and ModelNet.