[论文解读] PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation

Yisheng He, Wei Sun+4 more|arXiv (Cornell University)|Nov 11, 2019

Robot Manipulation and Learning参考文献 61被引用 37

一句话总结

PVN3D 提出了一种带实例分割的3D关键点投票网络，用于从单一 RGBD 图像估计6DoF位姿，在YCB-Video与LineMOD上优于以往方法。

ABSTRACT

In this work, we present a novel data-driven method for robust 6DoF object pose estimation from a single RGBD image. Unlike previous methods that directly regressing pose parameters, we tackle this challenging task with a keypoint-based approach. Specifically, we propose a deep Hough voting network to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner. Our method is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It allows us to fully utilize the geometric constraint of rigid objects with the extra depth information and is easy for a network to learn and optimize. Extensive experiments were conducted to demonstrate the effectiveness of 3D-keypoint detection in the 6D pose estimation task. Experimental results also show our method outperforms the state-of-the-art methods by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/PVN3D.git.

研究动机与目标

• 通过利用3D关键点而非直接姿态回归，推动基于RGB-D的鲁棒6DoF位姿估计。

提出的方法

• 开发一个深度3D关键点霍夫投票网络，学习每个点对选定3D关键点的平移偏移，并使用聚类来获得关键点。
• （保留原文中的结构及信息，逐条描述方法要点。）

实验结果

研究问题

• RQ1 基于3D关键点的投票能否在6DoF位姿估计中超越直接姿态回归和2D关键点方法？

主要发现

• 在YCB-Video和LineMOD数据集上，在ADD-S与ADD-S AUC指标上超越最先进方法。
• 实验表明，3D关键点表述在直接姿态回归和2D关键点方法中表现更优。
• 将3D关键点投票与实例语义分割联合训练，可提升姿态精度和分割质量。