[Paper Review] Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection

Wang Li-rui, Xiang Yu+1 more|arXiv (Cornell University)|Nov 22, 2019

Robot Manipulation and Learning39 references5 citations

TL;DR

This paper proposes an integrated optimization-based motion and grasp planner (OMG Planner) that jointly optimizes trajectories and grasps in cluttered environments by enabling online grasp synthesis and selection. By using iterative trajectory optimization with CHOMP and a probability-based grasp selection method, the approach improves planning success and robustness without requiring precomputed grasps, achieving 93% planning success and 84% execution success in evaluations.

ABSTRACT

In robot manipulation, planning the motion of a robot manipulator to grasp an object is a fundamental problem. A manipulation planner needs to generate a trajectory of the manipulator arm to avoid obstacles in the environment and plan an end-effector pose for grasping. While trajectory planning and grasp planning are often tackled separately, how to efficiently integrate the two planning problems remains a challenge. In this work, we present a novel method for joint motion and grasp planning. Our method integrates manipulation trajectory optimization with online grasp synthesis and selection, where we apply online learning techniques to select goal configurations for grasping, and introduce a new grasp synthesis algorithm to generate grasps online. We evaluate our planning approach and demonstrate that our method generates robust and efficient motion plans for grasping in cluttered scenes. Our video can be found at https://www.youtube.com/watch?v=LIcACf8YkGU .

Motivation & Objective

• To address the challenge of efficiently integrating motion planning and grasp planning in cluttered robotic manipulation tasks.
• To overcome limitations of fixed offline grasp sets and heuristic-based grasp selection in traditional methods.
• To develop a planner that dynamically selects and refines grasps during trajectory optimization to improve robustness and efficiency.
• To enable online grasp synthesis for arbitrary object shapes without relying on precomputed grasp candidates.
• To demonstrate improved performance in complex, cluttered scenes through joint optimization of motion and grasp planning.

Proposed method

• The OMG Planner formulates joint motion and grasp planning as an optimization problem, iteratively updating trajectories and grasps using CHOMP with goal set constraints.
• A probability distribution over the grasp set is estimated and updated online using a learning-based approach, enabling dynamic selection of the optimal grasp during planning.
• The C-Space ISF algorithm performs online grasp synthesis by refining grasp configurations through iterative surface fitting between the robot gripper and object in configuration space.
• Grasp selection is modeled as an online learning problem, where the algorithm learns from the interplay between current trajectories and grasp feasibility to improve goal selection.
• The method integrates existing grasp planning algorithms (e.g., GraspIt, simulated grasps) as initial grasp sets, which are then augmented and refined online.
• The framework supports any trajectory optimizer and is designed to be compatible with various motion planning methods beyond CHOMP.

Experimental results

Research questions

• RQ1 Can online grasp selection and synthesis improve the success rate and robustness of manipulation planning in cluttered scenes compared to fixed grasp sets?
• RQ2 How does integrating grasp synthesis during trajectory optimization affect planning efficiency and trajectory quality?
• RQ3 To what extent does online grasp refinement enhance grasp quality and execution success when starting from low-quality or suboptimal grasp candidates?
• RQ4 What is the impact of key design parameters such as discretization density (n) and smoothness weight (λ) on planner performance?
• RQ5 Can the proposed method achieve higher planning and execution success than baseline methods that use heuristic grasp selection or lack online refinement?

Key findings

• The OMG Planner achieved 93% planning success and 84% execution success across 100 scenes with 30 grasps per scene, outperforming CHOMP, FMT, and RRT-Connect.
• The method reduced average trajectory collision cost from 4.78 (CHOMP) to 3.92 and improved smoothness from 19.12 to 15.43, indicating better trajectory quality.
• Grasp refinement via C-Space ISF reduced grasp cost from 3.43 to 3.04 and increased success rate, especially when starting from high-quality simulated grasps.
• Ablation studies showed that increasing discretization (n) improved smoothness and planning success, while λ ≈ 0.1 provided the best balance between smoothness and collision avoidance.
• The integration of online grasp selection and synthesis significantly outperformed baseline methods that rely on heuristic grasp ranking or fixed grasp sets.
• The method demonstrated robust performance even with suboptimal initial grasp sets, highlighting the effectiveness of online refinement and selection.