[Paper Review] Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter

Michel Breyer, Jen Jen Chung+3 more|arXiv (Cornell University)|Jan 4, 2021

Robot Manipulation and Learning28 references44 citations

TL;DR

VGN predicts 6-DOF grasp quality, orientation, and width for every voxel in a TSDF volume, enabling real-time cluttered-scene grasping without explicit collision checks. It trains on synthetic data and transfers to real robots with GPU-accelerated inference in about 10 ms.

ABSTRACT

General robot grasping in clutter requires the ability to synthesize grasps that work for previously unseen objects and that are also robust to physical interactions, such as collisions with other objects in the scene. In this work, we design and train a network that predicts 6 DOF grasps from 3D scene information gathered from an on-board sensor such as a wrist-mounted depth camera. Our proposed Volumetric Grasping Network (VGN) accepts a Truncated Signed Distance Function (TSDF) representation of the scene and directly outputs the predicted grasp quality and the associated gripper orientation and opening width for each voxel in the queried 3D volume. We show that our approach can plan grasps in only 10 ms and is able to clear 92% of the objects in real-world clutter removal experiments without the need for explicit collision checking. The real-time capability opens up the possibility for closed-loop grasp planning, allowing robots to handle disturbances, recover from errors and provide increased robustness. Code is available at https://github.com/ethz-asl/vgn.

Motivation & Objective

• Motivate real-time 6-DOF grasp synthesis in cluttered scenes.
• Leverage full 3D scene information to implicitly handle collisions without explicit checks.
• Develop an end-to-end FCN that maps TSDF input to voxel-wise grasp quality, orientation, and width.
• Demonstrate transfer from synthetic training to real robotic setup without fine-tuning.

Proposed method

• Use a TSDF representation of the scene as input to a Fully Convolutional Network (FCN).
• Produce a three-headed output per voxel: grasp quality q, orientation r (as a quaternion), and gripper width w.
• Train end-to-end on synthetic cluttered grasp trials with a loss combining quality, rotation, and width terms.
• Incorporate symmetry handling for parallel-jaw grippers by allowing alternative ground-truth orientations in the rotation loss.
• Apply non-maxima suppression and voxel masking to extract top grasp candidates for execution.

Experimental results

Research questions

• RQ1 Can a voxel-wise, end-to-end FCN predict high-quality 6-DOF grasps directly from a TSDF of a cluttered scene?
• RQ2 Does using full 3D scene information improve collision avoidance and grasp success in clutter compared to methods relying on partial observations?
• RQ3 Is real-time grasp planning achievable with VGN, and can a model trained solely in simulation transfer to real robots without fine-tuning?

Key findings

• VGN achieves real-time planning, ~10 ms per inference on GPU.
• In simulated clutter experiments, VGN outperforms GPD in both success rate and objects cleared in several scenarios.
• With 0.9–0.95 grasp quality thresholds, VGN balances high success with more objects cleared.
• VGN transfers to real robot without additional fine-tuning, achieving 80% success and 92% objects cleared in 10 rounds.
• CPU-only real-robot inference is slower (~1.25 s) but still functional without GPUs.
• Some real-world failures are due to differences between physics simulation friction and real friction (e.g., cylindrical objects).