[Paper Review] DPOD: Dense 6D Pose Object Detector in RGB images

Sergey Zakharov, Ivan Shugurov+1 more|arXiv (Cornell University)|Feb 28, 2019

Advanced Neural Network Applications22 references32 citations

TL;DR

This paper proposes DPOD, a real-time, CNN-based method for joint 6D object pose estimation and detection in RGB images by predicting dense 2D-3D correspondences and multiclass masks. Unlike prior methods relying on 2D/3D bounding boxes, DPOD leverages these dense correspondences with PnP and RANSAC to achieve significantly improved pose accuracy, outperforming state-of-the-art approaches on both single- and multi-object benchmarks, even when trained only on synthetic data.

ABSTRACT

In this work we propose a new method for simultaneous object detection and 6DoF pose estimation. Unlike most recent techniques for CNN-based object detection and pose estimation, we do not base our approach on the common 2D counterparts, i.e. SSD and YOLO, but propose a new scheme. Instead of regressing 2D or 3D bounding boxes, we output full-sized 2D images containing multiclass object masks and dense 2D-3D correspondences. Having them at hand, a 6D pose is computed for each detected object using the PnP algorithm supplemented with RANSAC. This strategy allows for substantially better pose estimates due to a much higher number of relevant pose correspondences. Furthermore, the method is real-time capable, conceptually simple and not bound to any particular detection paradigms, such as R-CNN, SSD or YOLO. We test our method for single- and multiple-object pose estimation and compare the performance with the former state-of-the-art approaches. Moreover, we demonstrate how to use our pipeline when only synthetic renderings are available. In both cases, we outperform the former state-of-the-art by a large margin.

Motivation & Objective

• Address the limitation of existing 6D pose estimation methods that rely on sparse or bounding box-based supervision, which restricts pose accuracy.
• Develop a unified, real-time framework for simultaneous object detection and 6D pose estimation without dependence on specific detection architectures like R-CNN, SSD, or YOLO.
• Improve pose estimation accuracy by leveraging dense 2D-3D correspondences instead of sparse keypoint or box-based supervision.
• Enable effective training using only synthetic RGB renderings, reducing reliance on expensive real-world annotated data.

Proposed method

• Propose a novel detection head that outputs full-resolution multiclass segmentation masks and dense 2D-3D correspondences for each object instance.
• Use the predicted dense correspondences as input to a PnP algorithm with RANSAC to compute accurate 6D object poses.
• Design the network architecture to be detection-paradigm agnostic, enabling compatibility with various backbone networks and inference pipelines.
• Train the model end-to-end using a combination of segmentation and correspondence supervision, even when only synthetic renderings are available.
• Leverage the high density of correspondences to improve robustness and accuracy of pose estimation compared to sparse keypoint regression.
• Achieve real-time inference by avoiding complex region proposal networks and relying on dense prediction heads.

Experimental results

Research questions

• RQ1 Can dense 2D-3D correspondence prediction improve 6D object pose estimation accuracy compared to sparse keypoint or bounding box regression?
• RQ2 Is it possible to achieve state-of-the-art 6D pose estimation performance using only synthetic RGB renderings for training?
• RQ3 Can a detection-and-pose framework be designed to be independent of specific detection paradigms like R-CNN or YOLO?
• RQ4 How does the use of dense correspondences enhance pose estimation robustness and accuracy in both single- and multi-object scenarios?

Key findings

• DPOD achieves superior 6D pose estimation performance compared to the former state-of-the-art on both single- and multi-object benchmarks.
• The method significantly improves pose accuracy by leveraging dense 2D-3D correspondences, which provide more reliable and abundant data for PnP with RANSAC.
• DPOD is real-time capable, enabling practical deployment in dynamic environments.
• The framework generalizes well to real-world data even when trained exclusively on synthetic RGB renderings.
• The approach is not bound to any specific object detection paradigm, offering flexibility in model design and integration.
• The performance gain is substantial, with the paper stating a 'large margin' improvement over prior methods.