[Paper Review] Efficient State Abstraction using Object-centered Predicates for Manipulation Planning

Alejandro Agostini, Dongheui Lee|arXiv (Cornell University)|Jul 16, 2020

Robot Manipulation and Learning17 references4 citations

TL;DR

This paper proposes an object-centered state abstraction using object-centered predicates to enable more flexible and generalizable manipulation planning in robotics. By learning signal-symbol mappings via Gaussian Mixture Models (GMMs), it achieves higher accuracy (95%) and faster inference than kernel density estimation (KDE), enabling efficient, geometrically consistent planning without handcrafted predicates or external reference frames.

ABSTRACT

The definition of symbolic descriptions that consistently represent relevant geometrical aspects in manipulation tasks is a challenging problem that has received little attention in the robotic community. This definition is usually done from an observer perspective of a finite set of object relations and orientations that only satisfy geometrical constraints to execute experiments in laboratory conditions. This restricts the possible changes with manipulation actions in the object configuration space to those compatible with that particular external reference definitions, which greatly limits the spectrum of possible manipulations. To tackle these limitations we propose an object-centered representation that permits characterizing a much wider set of possible changes in configuration spaces than the traditional observer perspective counterpart. Based on this representation, we define universal planning operators for picking and placing actions that permits generating plans with geometric and force consistency in manipulation tasks. This object-centered description is directly obtained from the poses and bounding boxes of objects using a novel learning mechanisms that permits generating signal-symbols relations without the need of handcrafting these relations for each particular scenario.

Motivation & Objective

• To address the limitation of observer-centered state representations in manipulation planning, which restrict the set of possible object configuration changes.
• To develop a generalizable, object-centered representation that captures a wider spectrum of geometric relations without relying on arbitrary or handcrafted predicates.
• To enable efficient, real-time state abstraction in robotic cognitive architectures by learning signal-symbol mappings from object poses and bounding boxes.
• To improve geometric and force consistency in planning by identifying physically relevant surfaces for picking and placing actions.
• To replace memory-intensive, computation-heavy KDE-based mapping with a more adaptive, scalable GMM-based learning mechanism for predicate grounding.

Proposed method

• The approach defines symbolic predicates (e.g., on, under) based on object-centered geometric relations derived from object poses and bounding boxes.
• It employs Gaussian Mixture Models (GMMs) to learn the mapping between sensor signals (object geometry) and symbolic predicates, enabling online, real-time inference.
• The GMM-based mechanism adaptively models high-dimensional, sparsely populated configuration spaces, balancing bias and variance better than KDE.
• The method identifies physically relevant surfaces (e.g., contact or support surfaces) using neuroscientific insights on human manipulation, ensuring force consistency.
• It avoids handcrafting predicates by learning universal planning operators directly from data using GMMs, reducing dependency on scenario-specific definitions.
• The system is evaluated using 150 simulated manipulation scenarios, with performance measured via accuracy, misclassification rate, and computation time.

Experimental results

Research questions

• RQ1 Can an object-centered representation significantly expand the set of characterizable configuration space changes compared to observer-centered approaches?
• RQ2 Can a data-driven, GMM-based method for symbol grounding outperform traditional KDE-based approaches in accuracy and inference speed for robotic planning?
• RQ3 To what extent can object-centered predicates with learned signal-symbol mappings enable geometric and force consistency in manipulation planning without handcrafted rules?
• RQ4 How well does the GMM-based approach generalize across diverse object configurations in real-time planning scenarios?
• RQ5 Can the proposed method reduce the need for scenario-specific predicate engineering while maintaining high planning reliability?

Key findings

• The GMM-based approach achieved an average accuracy of 95% in predicate estimation, significantly outperforming the KDE baseline, which reached only 80% accuracy.
• The GMM method reduced misclassification rates by up to 50% compared to KDE, especially in low-density regions of the configuration space.
• Computation time for the GMM approach stabilized quickly after initial learning, while KDE required linearly increasing computation with each new scenario due to sample storage.
• For the predicate 'on', the GMM achieved a performance index of 0.67, compared to 0.39 for KDE, indicating superior classification reliability.
• For the predicate 'under', the GMM achieved a performance index of 0.95, far exceeding the KDE result of 0.60, demonstrating strong generalization in complex configurations.
• The GMM approach classified 95% of test instances correctly for 'under' and 67% for 'on', while KDE classified only 72% and 39% respectively, with significantly more unclassified samples.