[Paper Review] Object affordance as a guide for grasp-type recognition.

Naoki Wake, Daichi Saito+3 more|arXiv (Cornell University)|Feb 27, 2021

Robot Manipulation and Learning46 references4 citations

TL;DR

This paper proposes using object affordance—prior knowledge of likely grasp types per object—as a guide for convolutional neural network (CNN)-based grasp-type recognition. By filtering unlikely grasp candidates and emphasizing probable ones, object affordance improved recognition accuracy, especially in high-grasp-heterogeneity scenarios and under visual ambiguity from real or illusory objects.

ABSTRACT

Recognizing human grasping strategies is an important factor in robot teaching as these strategies contain the implicit knowledge necessary to perform a series of manipulations smoothly. This study analyzed the effects of object affordance-a prior distribution of grasp types for each object-on convolutional neural network (CNN)-based grasp-type recognition. To this end, we created datasets of first-person grasping-hand images labeled with grasp types and object names, and tested a recognition pipeline leveraging object affordance. We evaluated scenarios with real and illusory objects to be grasped, to consider a teaching condition in mixed reality where the lack of visual object information can make the CNN recognition challenging. The results show that object affordance guided the CNN in both scenarios, increasing the accuracy by 1) excluding unlikely grasp types from the candidates and 2) likely grasp types. In addition, the enhancing effect was more pronounced with high degrees of grasp-type heterogeneity. These results indicate the effectiveness of object affordance for guiding grasp-type recognition in robot teaching applications.

Motivation & Objective

• To improve grasp-type recognition in robot teaching by leveraging object affordance as prior knowledge.
• To address challenges in grasp recognition when visual object information is missing or ambiguous, such as in mixed reality environments.
• To evaluate whether object affordance enhances CNN-based grasp recognition in both real and illusory object scenarios.
• To investigate the impact of grasp-type heterogeneity on the effectiveness of affordance-guided recognition.

Proposed method

• Collected first-person grasping-hand image datasets labeled with grasp types and object names.
• Trained a CNN for grasp-type recognition using the collected image data.
• Integrated object affordance by pre-filtering CNN output candidates based on known grasp likelihoods per object.
• Evaluated the system on real objects and illusory objects (lacking visual cues) to simulate mixed reality teaching conditions.
• Used the affordance to exclude implausible grasp types and to prioritize likely ones during inference.
• Compared recognition accuracy with and without affordance guidance across varying degrees of grasp-type heterogeneity.

Experimental results

Research questions

• RQ1 How does object affordance influence grasp-type recognition accuracy in CNN-based systems?
• RQ2 Can object affordance improve recognition performance when visual object information is absent or ambiguous?
• RQ3 Does the effectiveness of affordance guidance vary with the level of grasp-type heterogeneity per object?
• RQ4 To what extent does affordance reduce the search space of possible grasp types during recognition?

Key findings

• Object affordance significantly improved grasp-type recognition accuracy by excluding unlikely grasp types from the candidate set.
• The integration of affordance enhanced recognition by emphasizing likely grasp types, particularly in complex scenarios.
• The performance gain was more pronounced in objects with high grasp-type heterogeneity, indicating greater benefit in complex recognition tasks.
• The method remained effective even when visual object information was absent, such as in illusory object scenarios, demonstrating robustness in mixed reality teaching settings.