[Paper Review] DcnnGrasp: Towards Accurate Grasp Pattern Recognition with Adaptive Regularizer Learning

Xiaoqin Zhang, Ziwei Huang+3 more|arXiv (Cornell University)|May 11, 2022

Robot Manipulation and Learning4 citations

TL;DR

This paper proposes DcnnGrasp, a dual-branch CNN that jointly learns object category classification and grasp pattern recognition using a novel joint cross-entropy with adaptive regularizer (JCEAR) loss function. By leveraging category information as an auxiliary task and optimizing collaborative learning, DcnnGrasp achieves a 15% higher global accuracy than the second-best method on unseen objects in the RGB-D Object dataset, demonstrating strong generalization and robustness to missing grasp labels.

ABSTRACT

The task of grasp pattern recognition aims to derive the applicable grasp types of an object according to the visual information. Current state-of-the-art methods ignore category information of objects which is crucial for grasp pattern recognition. This paper presents a novel dual-branch convolutional neural network (DcnnGrasp) to achieve joint learning of object category classification and grasp pattern recognition. DcnnGrasp takes object category classification as an auxiliary task to improve the effectiveness of grasp pattern recognition. Meanwhile, a new loss function called joint cross-entropy with an adaptive regularizer is derived through maximizing a posterior, which significantly improves the model performance. Besides, based on the new loss function, a training strategy is proposed to maximize the collaborative learning of the two tasks. The experiment was performed on five household objects datasets including the RGB-D Object dataset, Hit-GPRec dataset, Amsterdam library of object images (ALOI), Columbia University Image Library (COIL-100), and MeganePro dataset 1. The experimental results demonstrated that the proposed method can achieve competitive performance on grasp pattern recognition with several state-of-the-art methods. Specifically, our method even outperformed the second-best one by nearly 15% in terms of global accuracy for the case of testing a novel object on the RGB-D Object dataset.

Motivation & Objective

• To address the challenge of grasp pattern recognition on unseen objects, especially when grasp labels are scarce or ambiguous.
• To improve model generalization by incorporating object category information as an auxiliary signal in grasp recognition.
• To develop a training strategy that maximizes collaborative learning between category classification and grasp pattern recognition.
• To enhance robustness to missing or incomplete grasp labeling, a common issue in real-world applications.
• To overcome limitations of existing methods that ignore category-level priors, which humans naturally use for grasp prediction.

Proposed method

• DcnnGrasp employs a dual-branch CNN architecture where one branch classifies object categories and the other predicts grasp types, sharing early convolutional features.
• A novel loss function, joint cross-entropy with adaptive regularizer (JCEAR), is derived by maximizing the posterior probability, promoting joint optimization of both tasks.
• The adaptive regularizer dynamically adjusts the loss weighting between category and grasp tasks based on model uncertainty, improving training stability and performance.
• A dedicated training strategy is designed to maximize the collaborative learning between the two tasks, ensuring that category knowledge enhances grasp recognition.
• The method is trained on two newly constructed dual-label datasets derived from RGB-D Object and Hit-GPRec, with both category and grasp labels annotated.

Experimental results

Research questions

• RQ1 Can incorporating object category information as an auxiliary task significantly improve grasp pattern recognition performance, especially on unseen objects?
• RQ2 How does a joint loss function with adaptive regularization enhance the collaborative learning between category classification and grasp recognition?
• RQ3 To what extent can DcnnGrasp generalize to novel objects not seen during training, particularly under challenging conditions like size or structural confusion?
• RQ4 How robust is the model when grasp labels are missing or sparse in the training data?
• RQ5 Can the proposed JCEAR loss and training strategy outperform existing state-of-the-art methods in both labeled and partially labeled scenarios?

Key findings

• On the RGB-D Object dataset, DcnnGrasp achieved a global accuracy of over 94% in unseen object testing, outperforming the second-best method by nearly 15%.
• On the Hit-GPRec dataset, DcnnGrasp maintained performance above 99% across all evaluation metrics, demonstrating high robustness to dataset-specific noise like shadows.
• Even with only one labeled grasp example per category, DcnnGrasp achieved 90% and 95% global accuracy on the RGB-D Object and Hit-GPRec datasets, respectively, showing strong generalization under low-shot conditions.
• The ablation study confirmed that the JCEAR-based training strategy significantly improves performance, especially in handling unseen objects and label scarcity.
• DcnnGrasp showed superior robustness to missing grasp labels: while CnnGrasp’s performance dropped by 10–15% across different label proportions, DcnnGrasp’s performance varied by less than 2%.