[Paper Review] Robust 6D Object Pose Estimation by Learning RGB-D Features

Meng Tian, Liang Pan+2 more|arXiv (Cornell University)|Feb 29, 2020

Robot Manipulation and Learning42 references4 citations

TL;DR

This paper proposes a novel deep learning method for robust 6D object pose estimation using RGB-D features, addressing the local-optimum problem in rotation regression via a discrete-continuous formulation with rotation anchors and uncertainty-aware deviation prediction. The method achieves state-of-the-art performance on LINEMOD (92.8% ADD) and YCB-Video (83.8% ADD), outperforming prior RGB-D methods by up to 6.6% on small, textureless objects and demonstrating superior robustness to occlusions and lighting variations through separate, constrained regression for rotation and translation.

ABSTRACT

Accurate 6D object pose estimation is fundamental to robotic manipulation and grasping. Previous methods follow a local optimization approach which minimizes the distance between closest point pairs to handle the rotation ambiguity of symmetric objects. In this work, we propose a novel discrete-continuous formulation for rotation regression to resolve this local-optimum problem. We uniformly sample rotation anchors in SO(3), and predict a constrained deviation from each anchor to the target, as well as uncertainty scores for selecting the best prediction. Additionally, the object location is detected by aggregating point-wise vectors pointing to the 3D center. Experiments on two benchmarks: LINEMOD and YCB-Video, show that the proposed method outperforms state-of-the-art approaches. Our code is available at https://github.com/mentian/object-posenet.

Motivation & Objective

• To address the local-optimum problem in 6D object pose estimation caused by symmetric objects and ambiguous rotation predictions.
• To improve robustness to occlusions, background clutter, and varying lighting conditions in RGB-D-based pose estimation.
• To develop a unified deep network that separately regresses rotation and translation using densely extracted RGB-D features.
• To introduce uncertainty-aware prediction for rotation anchors to improve selection of the best pose hypothesis.
• To achieve real-time inference speed suitable for robotic manipulation applications.

Proposed method

• The method uses uniformly sampled rotation anchors across SO(3) to discretize the rotation space, enabling localized regression.
• For each anchor, the network predicts a constrained deviation vector to the target rotation, reducing the risk of local optima.
• Uncertainty scores are predicted for each anchor and used to select the most reliable rotation estimate during inference.
• Translation is estimated by regressing a unit vector at each point toward the 3D object center, followed by a RANSAC-based voting layer to aggregate predictions.
• The network is trained end-to-end using a combination of rotation and translation loss functions, with uncertainty supervision via self-supervised conditional probability maximization.
• RGB-D features are densely extracted using a CNN backbone, fusing color and geometric information for improved feature representation.

Experimental results

Research questions

• RQ1 Can a discrete-continuous formulation for rotation regression reduce the local-optimum problem in 6D pose estimation?
• RQ2 How does uncertainty-aware prediction improve robustness in rotation estimation under symmetric object ambiguity?
• RQ3 Can separate regression of rotation and translation outperform joint optimization in terms of accuracy and robustness?
• RQ4 To what extent does dense RGB-D feature extraction improve performance on textureless or occluded objects?
• RQ5 Can the proposed method achieve real-time inference while maintaining high accuracy on standard benchmarks?

Key findings

• The proposed method achieves 92.8% accuracy on the LINEMOD dataset using the ADD metric, significantly outperforming the previous state-of-the-art method (86.3%).
• On the more challenging YCB-Video dataset, the method achieves 83.8% accuracy in the ADD metric, a 4.6% improvement over the prior SOTA method Per-Pixel DF (79.2%).
• The method shows a 13.6% higher accuracy on the small, textureless 'holepuncher' object compared to baseline methods, demonstrating robustness to low-texture challenges.
• The method achieves a 6.6% higher detection accuracy than PVNet and Per-Pixel DF on LINEMOD, particularly excelling on symmetric and small objects.
• The network runs at approximately 14 FPS on a single GPU (0.07s per frame), enabling real-time inference suitable for robotic applications.
• Qualitative results show that the method avoids local-optimum predictions common in PoseCNN and DenseFusion, especially on symmetric objects, due to the discrete-continuous rotation formulation.