[Paper Review] Stereo Vision Based Single-Shot 6D Object Pose Estimation for Bin-Picking by a Robot Manipulator

Yoshihiro Nakano|arXiv (Cornell University)|May 28, 2020

Robot Manipulation and Learning28 references4 citations

TL;DR

This paper proposes a stereo vision-based single-shot 6D object pose estimation method for robotic bin-picking using a novel Stereo Grid Attention module that focuses disparity computation only on detected objects, enabling fast and accurate pose estimation. The method processes 1024×1024 images in 75ms on a Jetson AGX Xavier and achieves 89.1% success rate for DC motor parts using synthetic data and real-world testing with an electromagnetic gripper.

ABSTRACT

We propose a fast and accurate method of 6D object pose estimation for bin-picking of mechanical parts by a robot manipulator. We extend the single-shot approach to stereo vision by application of attention architecture. Our convolutional neural network model regresses to object locations and rotations from either a left image or a right image without depth information. Then, a stereo feature matching module, designated as Stereo Grid Attention, generates stereo grid matching maps. The important point of our method is only to calculate disparity of the objects found by the attention from stereo images, instead of calculating a point cloud over the entire image. The disparity value is then used to calculate the depth to the objects by the principle of triangulation. Our method also achieves a rapid processing speed of pose estimation by the single-shot architecture and it is possible to process a 1024 x 1024 pixels image in 75 milliseconds on the Jetson AGX Xavier implemented with half-float model. Weakly textured mechanical parts are used to exemplify the method. First, we create original synthetic datasets for training and evaluating of the proposed model. This dataset is created by capturing and rendering numerous 3D models of several types of mechanical parts in virtual space. Finally, we use a robotic manipulator with an electromagnetic gripper to pick up the mechanical parts in a cluttered state to verify the validity of our method in an actual scene. When a raw stereo image is used by the proposed method from our stereo camera to detect black steel screws, stainless screws, and DC motor parts, i.e., cases, rotor cores and commutator caps, the bin-picking tasks are successful with 76.3%, 64.0%, 50.5%, 89.1% and 64.2% probability, respectively.

Motivation & Objective

• To enable fast and accurate 6D object pose estimation in cluttered bin-picking scenarios using stereo vision.
• To reduce computational cost by focusing disparity computation only on detected objects rather than the full image.
• To develop a single-shot deep learning framework that regresses pose from left or right stereo images without depth input.
• To validate the method on weakly textured mechanical parts using synthetic datasets and real-world robotic manipulation.
• To achieve real-time performance suitable for deployment on edge devices like the Jetson AGX Xavier.

Proposed method

• A convolutional neural network regresses 6D object pose (position and orientation) from a single left or right stereo image without requiring depth maps.
• An attention mechanism identifies salient object regions, restricting disparity computation to only these regions to improve efficiency.
• The Stereo Grid Attention module generates stereo grid matching maps to compute disparity only for detected objects.
• Disparity values are converted to depth using triangulation principles, enabling 3D pose estimation.
• A synthetic dataset of 3D mechanical part models rendered in virtual environments is created for training and evaluation.
• The method is deployed on a robotic manipulator with an electromagnetic gripper to perform real-world bin-picking tasks.

Experimental results

Research questions

• RQ1 Can a single-shot stereo vision approach achieve accurate 6D object pose estimation in cluttered bin-picking environments?
• RQ2 Can attention-based disparity computation improve efficiency by focusing only on detected objects?
• RQ3 Can synthetic data effectively train a model for real-world bin-picking of weakly textured mechanical parts?
• RQ4 What is the real-time performance of the method on edge hardware like the Jetson AGX Xavier?
• RQ5 How does the method perform on diverse mechanical parts with varying textures and shapes in real-world scenarios?

Key findings

• The method processes 1024×1024 pixel stereo images in 75 milliseconds on a Jetson AGX Xavier using a half-precision model.
• The system achieved a 76.3% success rate in picking black steel screws from a cluttered bin.
• Stainless steel screws were picked successfully in 64.0% of attempts.
• DC motor case parts were picked with 50.5% success rate, rotor cores with 89.1%, and commutator caps with 64.2%.
• The use of synthetic data enabled effective training for real-world deployment on mechanical parts with weak texture.
• The attention-based disparity computation significantly reduced processing load by avoiding full-image disparity estimation.