[Paper Review] Learning Based Industrial Bin-picking Trained with Approximate Physics Simulator

Ryo Matsumura, Kensuke Harada+2 more|arXiv (Cornell University)|May 23, 2018

Robot Manipulation and Learning7 references3 citations

TL;DR

This paper proposes a learning-based industrial bin-picking system using a convolutional neural network (CNN) trained on synthetic data generated by an approximate physics simulator to accelerate data collection. Despite using shape approximations in collision detection for simulation efficiency, the method maintains high prediction accuracy by using exact 3D models to generate depth image inputs to the CNN, effectively relaxing the impact of approximation on performance.

ABSTRACT

In this research, we tackle the problem of picking an object from randomly stacked pile. Since complex physical phenomena of contact among objects and fingers makes it difficult to perform the bin-picking with high success rate, we consider introducing a learning based approach. For the purpose of collecting enough number of training data within a reasonable period of time, we introduce a physics simulator where approximation is used for collision checking. In this paper, we first formulate the learning based robotic bin-picking by using CNN (Convolutional Neural Network). We also obtain the optimum grasping posture of parallel jaw gripper by using CNN. Finally, we show that the effect of approximation introduced in collision checking is relaxed if we use exact 3D model to generate the depth image of the pile as an input to CNN.

Motivation & Objective

• Address the challenge of collecting sufficient training data for learning-based robotic bin-picking within a feasible timeframe.
• Overcome the computational bottleneck of simulating complex physical interactions in randomly stacked object piles with irregular shapes.
• Minimize the negative impact of shape approximation in physics simulation on the accuracy of CNN-based success prediction for grasping.
• Develop a method that enables high-accuracy grasping policy learning even when collision checks use simplified object shapes.
• Demonstrate that using exact 3D models for depth image generation mitigates the performance degradation caused by shape approximation in simulation.

Proposed method

• Employ a physics simulator with shape approximation for collision checking to reduce simulation time during data generation.
• Generate synthetic depth images of object piles using original, high-fidelity 3D models to preserve geometric fidelity in network input.
• Train a CNN to predict the success probability of grasping an object given a depth image and a candidate grasping pose.
• Optimize the grasping pose by selecting the one with the highest predicted success rate from the CNN output.
• Use a rectangular approximation of convex decomposed object parts for collision detection while retaining original geometry for rendering depth images.
• Evaluate the robustness of the trained CNN by analyzing prediction errors in cases where approximation leads to false positives in simulation.

Experimental results

Research questions

• RQ1 To what extent does shape approximation in physics simulation degrade the performance of a CNN-based bin-picking policy?
• RQ2 Can the use of exact 3D models for depth image generation compensate for inaccuracies introduced by shape approximation in collision detection?
• RQ3 What is the maximum acceptable rate of approximation in training data that still allows reliable generalization to real-world picking tasks?
• RQ4 How does the CNN distinguish between realistic and unrealistic grasps caused by simulation approximation?
• RQ5 Can a learning-based approach maintain high success rates in real-world bin-picking despite using approximate physics during training?

Key findings

• The CNN model successfully identifies the optimal grasping pose with high success rate, as validated on 20 verification test cases.
• Even when physics simulation uses approximated shapes, the CNN maintains reliable prediction accuracy by using exact 3D models for depth image input.
• In 15 out of 200 simulated cases, the robot successfully picked an object due to approximation-induced false positives, yet the CNN predicted low success rates (below 0.5) in most such cases.
• When the rate of using approximated objects in training was kept below 30%, the CNN correctly predicted picking success or failure with high accuracy.
• The method enables effective data collection via fast approximate simulation while preserving generalization capability through exact depth image input.
• The approach achieves a balance between simulation speed and prediction reliability, making large-scale training feasible for industrial bin-picking applications.