[论文解读] Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation

Pete Florence, Lucas Manuelli+1 more|arXiv (Cornell University)|Jun 22, 2018

Robot Manipulation and Learning参考文献 32被引用 99

一句话总结

Dense Object Nets 在自监督方式下学习像素级密集视觉描述符，以支持对刚性和非刚性对象的操作，具备多目标和类别泛化能力，实现对特定点的抓取以及跨实例/类别的转移。

ABSTRACT

What is the right object representation for manipulation? We would like robots to visually perceive scenes and learn an understanding of the objects in them that (i) is task-agnostic and can be used as a building block for a variety of manipulation tasks, (ii) is generally applicable to both rigid and non-rigid objects, (iii) takes advantage of the strong priors provided by 3D vision, and (iv) is entirely learned from self-supervision. This is hard to achieve with previous methods: much recent work in grasping does not extend to grasping specific objects or other tasks, whereas task-specific learning may require many trials to generalize well across object configurations or other tasks. In this paper we present Dense Object Nets, which build on recent developments in self-supervised dense descriptor learning, as a consistent object representation for visual understanding and manipulation. We demonstrate they can be trained quickly (approximately 20 minutes) for a wide variety of previously unseen and potentially non-rigid objects. We additionally present novel contributions to enable multi-object descriptor learning, and show that by modifying our training procedure, we can either acquire descriptors which generalize across classes of objects, or descriptors that are distinct for each object instance. Finally, we demonstrate the novel application of learned dense descriptors to robotic manipulation. We demonstrate grasping of specific points on an object across potentially deformed object configurations, and demonstrate using class general descriptors to transfer specific grasps across objects in a class.

研究动机与目标

• 开发用于操作的自监督密集视觉描述符表示，任务无关，适用于刚性和非刚性对象。
• 实现对未见对象和对象类别的密集描述符的快速学习（约20分钟）。
• 创建多对象的不同描述符技术，以及可跨类别泛化或保持实例特定的描述符。
• 演示利用学习到的密集描述符来抓取对象上的特定点，以及在实例或类别之间转移抓取。

提出的方法

• 使用密集像素级对比损失来训练一个将 RGB 图像映射到 W×H×D 描述符空间的映射 f。
• 通过 3D 稠密重建从 RGBD 视频中构造匹配和非匹配项，实现无需人工标签的自监督学习。
• 应用来自 3D 变化检测的面向对象的掩模，将训练聚焦在对象上并忽略背景。
• 引入背景域随机化和困难负样本缩放，以提高跨场景的一致性和描述符精度。
• 扩展到多对象密集描述符，使用跨对象损失、场景内直接多对象训练，以及合成多对象场景的组合。

实验结果

研究问题

• RQ1 通过自监督学习得到的密集像素级描述符是否能够在不同视角和变形下提供一致的对象表示？
• RQ2 如何在不影响密集描述符的实例特定精度的前提下实现多对象和类别泛化？
• RQ3 在多大程度上，学习得到的密集描述符能够实现跨配置抓取对象特定点等操作，以及在对象实例或类别之间转移抓取？

主要发现

• 描述符在多样化对象的显著变形下保持一致，即使是低纹理物体。
• 跨对象损失和更大的描述符空间使得多个对象可以有不同的描述符区域，而不牺牲性能。
• 描述符能够在类别之间泛化（如帽子、鞋子、杯子）并在同一类别的未见实例之间转移抓取。
• 不同对象技术能够在混乱环境和多对象配置下抓取对象的特定点。
• 背景域随机化和方向随机化提升描述符鲁棒性，尤其在小数据集上。