[Paper Review] Grasp2Vec: Learning Object Representations from Self-Supervised Grasping

Eric Jang, Coline Devin+2 more|arXiv (Cornell University)|Nov 16, 2018

Robot Manipulation and LearningEngineering28 references74 citations

TL;DR

Grasp2Vec learns object-centric embeddings from self-supervised robotic grasping by enforcing that the scene difference after grasping equals the embedding of the grasped object, enabling localization, instance detection, and goal-conditioned grasping without labels.

ABSTRACT

Well structured visual representations can make robot learning faster and can improve generalization. In this paper, we study how we can acquire effective object-centric representations for robotic manipulation tasks without human labeling by using autonomous robot interaction with the environment. Such representation learning methods can benefit from continuous refinement of the representation as the robot collects more experience, allowing them to scale effectively without human intervention. Our representation learning approach is based on object persistence: when a robot removes an object from a scene, the representation of that scene should change according to the features of the object that was removed. We formulate an arithmetic relationship between feature vectors from this observation, and use it to learn a representation of scenes and objects that can then be used to identify object instances, localize them in the scene, and perform goal-directed grasping tasks where the robot must retrieve commanded objects from a bin. The same grasping procedure can also be used to automatically collect training data for our method, by recording images of scenes, grasping and removing an object, and recording the outcome. Our experiments demonstrate that this self-supervised approach for tasked grasping substantially outperforms direct reinforcement learning from images and prior representation learning methods.

Motivation & Objective

• Promote automatic, self-supervised learning of object-centric scene representations for robotic manipulation.

Proposed method

• Embed scenes and grasped objects with CNNs based on ResNet-50 to produce phi_s and phi_o embeddings.
• Impose an arithmetic constraint: phi_s(s_pre) - phi_s(s_post) ≈ phi_o(o) to capture object identity and persistence.
• Train using the n-pairs loss to align the scene-difference embedding with the object embedding and separate negatives.
• Use the learned Grasp2Vec embeddings to localize objects via spatial heatmaps and to condition a goal-directed grasping policy via Q-learning.
• Data for training is collected autonomously through grasping episodes (s_pre, s_post, o).

Experimental results

Research questions

• RQ1 Can a self-supervised embedding learned from grasping capture object identity and the set of objects in a scene?
• RQ2 Can Grasp2Vec embeddings localize and distinguish object instances without labeled data?
• RQ3 Can a goal-conditioned grasping policy be trained using rewards derived from Grasp2Vec embeddings without manual annotations?
• RQ4 How well does Grasp2Vec generalize to unseen objects, both in simulation and the real world?

Key findings

• Grasp2Vec retrieval accuracy: 88% (sim seen), 64% (sim novel), 89% (real seen), 88% (real novel).
• Grasp2Vec localization accuracy: 96% (sim seen), 77% (sim novel), 83% (real seen), 81% (real novel).
• Localization with ImageNet features performs significantly worse (15-18% range) for the same task.
• In simulation, instance grasping with Grasp2Vec-based ES rewards reaches 78-83% on seen objects and 53-59% on unseen objects depending on ablations.
• Real-world instance grasping using localization plus indiscriminate grasping achieves 80.8% on training objects and 62.9% on test objects.
• Composite goals via additive Grasp2Vec embeddings enable some multi-object goal behaviors in simulation (e.g., 51.9% seen, 42.9% unseen for certain composite goals).