[Paper Review] S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes

Yuzhe Qin, Rui Chen+4 more|arXiv (Cornell University)|Oct 31, 2019

Robot Manipulation and LearningEngineering32 references55 citations

TL;DR

The paper presents S4G, a single-shot 6-DoF grasp proposal network that regresses SE(3) grasp poses from a single-view depth point cloud, trained on synthetic cluttered scenes, and demonstrates superior performance over state-of-the-art methods.

ABSTRACT

Grasping is among the most fundamental and long-lasting problems in robotics study. This paper studies the problem of 6-DoF(degree of freedom) grasping by a parallel gripper in a cluttered scene captured using a commodity depth sensor from a single viewpoint. We address the problem in a learning-based framework. At the high level, we rely on a single-shot grasp proposal network, trained with synthetic data and tested in real-world scenarios. Our single-shot neural network architecture can predict amodal grasp proposal efficiently and effectively. Our training data synthesis pipeline can generate scenes of complex object configuration and leverage an innovative gripper contact model to create dense and high-quality grasp annotations. Experiments in synthetic and real environments have demonstrated that the proposed approach can outperform state-of-the-arts by a large margin.

Motivation & Objective

• Address 6-DoF grasping in cluttered scenes from a single-view depth sensor without object category assumptions.
• Develop a single-shot grasp proposal network that regresses SE(3) grasps directly for efficiency and accuracy.
• Create a physically-plausible, dense synthetic training dataset with a novel gripper contact model.
• Ensure robustness to partial, noisy depth data through amodal grasp prediction and scene analysis.

Proposed method

• Propose a single-shot grasp proposal network based on PointNet++ that assigns each point a 6-DoF grasp pose and a quality score.
• Represent 6-DoF rotations with a continuous 6D rotation representation and use a rotation loss that accounts for gripper symmetry.
• Develop a gripper contact model using force-closure concepts to generate antipodal and occupancy-based grasp scores.
• Synthesize physically plausible cluttered scenes in MuJoCo with V-HACD to create realistic training data and render noisy depth views.
• Combine local and global geometry via the network to predict collision-free grasps in dense clutter, followed by non-maximum suppression and probabilistic grasp sampling for execution.

Experimental results

Research questions

• RQ1 Can a single-shot SE(3) grasp detector predict viable, collision-free 6-DoF grasps from a single-view, partial point cloud in cluttered scenes?
• RQ2 Does learning to regress grasp proposals directly from scene data outperform sampling-based approaches in clutter and noise?
• RQ3 How does synthetic data with a realistic gripper contact model generalize to real-world cluttered manipulation tasks?
• RQ4 What is the impact of 6-DoF (SE(3)) grasp prediction versus traditional 3/4-DoF approaches in terms of success in dense scenes?

Key findings

• The method regresses 6-DoF grasps directly for each point and achieves robust performance in cluttered scenes.
• In simulation, the approach outperforms baselines such as GPD variants and PointNetGPD in antipodal score and collision-free grasps under noise.
• Robotic experiments show higher success and completion rates with S4G (e.g., 77.1% success, 92.5% completion) and faster inference (approximately 12.6 ms per grasp) compared with baselines.
• A recall of 63.38% indicates improved capability of SE(3) grasps over 3/4-DoF grasps under varying scene densities.
• The pipeline efficiently generates executable grasps through NMS and probabilistic sampling, enabling real-time or near-real-time manipulation in cluttered environments.