[Paper Review] Generative grasp synthesis from demonstration using parametric mixtures

Ermano Arruda, Claudio Zito+3 more|arXiv (Cornell University)|Jun 27, 2019

Robot Manipulation and Learning20 references4 citations

TL;DR

This paper proposes a parametric Gaussian Mixture Model (GMM) formulation for generative grasp synthesis from demonstration, replacing non-parametric Kernel Density Estimation (KDE) to achieve faster inference and improved grasp success rates. The method models grasp configurations in SE(3) with quaternion-based rotations and curvature features, using EM learning and a product-of-experts objective that incorporates task-specific constraints, yielding a 10%+ success rate gain and ~5× faster runtime in simulation and robot deployment.

ABSTRACT

We present a parametric formulation for learning generative models for grasp synthesis from a demonstration. We cast new light on this family of approaches, proposing a parametric formulation for grasp synthesis that is computationally faster compared to related work and indicates better grasp success rate performance in simulated experiments, showing a gain of at least 10% success rate (p < 0.05) in all the tested conditions. The proposed implementation is also able to incorporate arbitrary constraints for grasp ranking that may include task-specific constraints. Results are reported followed by a brief discussion on the merits of the proposed methods noted so far.

Motivation & Objective

• Address the computational inefficiency of non-parametric KDE-based grasp synthesis, which scales poorly with training data size due to O(KN) likelihood evaluation complexity.
• Improve grasp success rates in simulated and real-world conditions by replacing KDE with a parametric mixture model that generalizes better under sensor noise.
• Enable efficient integration of task-specific constraints—such as kinematic feasibility and axis alignment—into the grasp synthesis pipeline for real robot deployment.
• Demonstrate that a GMM-based approach can achieve superior performance and significantly faster inference compared to KDE, making real-time grasp synthesis feasible.

Proposed method

• Represent grasp configurations using SE(3) transformations combined with principal curvature features (k₁, k₂) from object point clouds, forming a 7+n-dimensional input space.
• Use logarithmic and exponential maps to convert quaternions into angular velocity vectors (ω ∈ ℝ³) for linear Gaussian mixture modeling in the tangent space.
• Learn a parametric GMM via Expectation Maximization (EM) on demonstrated grasp data, with a fixed, small number of kernels (K ≤ 10), ensuring fast likelihood evaluation independent of training set size.
• Formulate a product-of-experts objective function (Eq. 17) that combines grasp quality, likelihood, and task-specific constraints, enabling ranking and optimization of candidate grasps.
• Apply simulated annealing to refine the top 10 grasps from 200 samples, improving performance under noisy sensing conditions.
• Integrate kinematic constraints directly into the evaluation step using Eq. 21, ensuring only feasible grasps are considered during optimization.

Experimental results

Research questions

• RQ1 Can a parametric mixture model (GMM) outperform non-parametric KDE in terms of grasp success rate and inference speed for generative grasp synthesis from demonstration?
• RQ2 How does the proposed method perform under varying sensor noise conditions, and does optimization significantly improve results in noisy settings?
• RQ3 To what extent can task-specific constraints—such as kinematic feasibility and axis alignment—be effectively integrated into the grasp synthesis pipeline using the proposed formulation?
• RQ4 Is the GMM-based approach fast enough to support real-time or near real-time grasp synthesis on a physical robot platform?

Key findings

• The GMM-based method achieved a statistically significant improvement in grasp success rate, with a minimum gain of 10% (p < 0.05) across all tested conditions compared to KDE.
• The total runtime for grasp synthesis was approximately 5 times faster with the GMM-based method, primarily due to reduced kernel count and O(K) likelihood evaluation complexity.
• Under noisy conditions, the final optimization step (simulated annealing) significantly improved grasp performance, indicating its importance in real-world scenarios with imperfect sensing.
• The method successfully deployed on the Boris robot platform, demonstrating feasibility of real-world application with integrated kinematic constraints.
• The GMM-based approach maintained high performance even with a small number of kernels (K_Q = 5), showing effective density approximation with minimal computational cost.
• The product-of-experts formulation enabled seamless integration of task-specific constraints into the grasp ranking process, enhancing practical deployment robustness.