[Paper Review] Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

Jeffrey Mahler, Matthew Matl+4 more|arXiv (Cornell University)|Sep 19, 2017

Robot Manipulation and Learning18 citations

TL;DR

This paper proposes Dex-Net 3.0, a deep learning framework that uses a novel compliant suction contact model to predict robust vacuum suction grasp targets in 3D point clouds. By modeling seal formation and wrench resistance under pose, material, and disturbance perturbations, it achieves 98% success on basic objects, 82% on typical objects, and 58% on adversarial objects in physical robot trials using a GQ-CNN trained on 2.8 million synthetic grasps.

ABSTRACT

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98$\%$, 82$\%$, and 58$\%$ respectively, improving to 81$\%$ in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net .

Motivation & Objective

• To address the lack of robust suction grasp planning for complex, non-planar objects in industrial robotics.
• To develop a physics-based model that evaluates both seal formation feasibility and resistance to external wrenches under uncertainty.
• To generate a large-scale synthetic dataset of 2.8 million point clouds with labeled grasp robustness for training.
• To train a GQ-CNN that classifies high-quality suction grasp targets in real-world point clouds.
• To validate the method on a physical robot system across diverse object categories, including adversarial geometries.

Proposed method

• A quasi-static spring model with perimeter, flexion, and cone springs is used to simulate suction cup deformation and test seal feasibility at a candidate grasp point.
• The model computes wrench resistance using a contact wrench basis derived from the ring of contact between the cup and object surface.
• Robustness is evaluated via Monte Carlo sampling over perturbations in object pose, gripper pose, friction coefficient, and external wrenches (e.g., gravity).
• A grasp success probability is modeled as a Bernoulli distribution based on whether the grasp resists a randomly sampled disturbing wrench under perturbed states.
• The GQ-CNN is trained on synthetic depth images with grasp thumbnails centered on predicted targets, using binary labels derived from thresholded robustness scores.
• The system uses offscreen rendering with noise models for depth images and probabilistic sampling for pose and wrench uncertainty.

Experimental results

Research questions

• RQ1 Can a physics-based compliant contact model accurately predict whether a suction cup can form a seal on non-planar surfaces?
• RQ2 How can wrench resistance robustness be quantified under realistic uncertainties in pose, friction, and external forces?
• RQ3 Can a deep learning model trained on synthetic data generalize to real-world suction grasping on diverse object geometries?
• RQ4 What is the performance gap between heuristic methods and learned robust grasp policies on novel, complex, and adversarial objects?
• RQ5 Does including adversarial object data in training improve generalization to challenging geometries?

Key findings

• The proposed compliant suction contact model successfully predicts seal formation feasibility on non-planar surfaces, enabling robust grasp planning beyond simple planar heuristics.
• The system achieves a 98% success rate on Basic objects (prismatic or cylindrical) in physical trials, significantly outperforming heuristic methods.
• On Typical objects with complex geometry, the method achieves 82% success, demonstrating strong generalization beyond simple shapes.
• On Adversarial objects with few viable suction points, the success rate is 58%, which improves to 81% when the training set includes only adversarial objects.
• The GQ-CNN trained on Dex-Net 3.0 generalizes well to novel objects and outperforms baseline heuristics that target centroids or planar surfaces.
• The synthetic dataset of 2.8 million point clouds with robustness labels enables effective transfer learning for real-world robotic suction grasping.