[Paper Review] 6DoF Object Pose Estimation via Differentiable Proxy Voting Loss

Xin Yu, Zheyu Zhuang+2 more|arXiv (Cornell University)|Feb 10, 2020

Robot Manipulation and Learning17 references22 citations

TL;DR

This paper proposes a differentiable proxy voting loss (DPVL) for 6DoF object pose estimation that improves vector-field regression by incorporating pixel-to-keypoint distances, reducing hypothesis deviation from inaccurate direction vectors. DPVL enables end-to-end training, achieving state-of-the-art performance and faster convergence on LINEMOD and Occlusion LINEMOD datasets.

ABSTRACT

Estimating a 6DOF object pose from a single image is very challenging due to occlusions or textureless appearances. Vector-field based keypoint voting has demonstrated its effectiveness and superiority on tackling those issues. However, direct regression of vector-fields neglects that the distances between pixels and keypoints also affect the deviations of hypotheses dramatically. In other words, small errors in direction vectors may generate severely deviated hypotheses when pixels are far away from a keypoint. In this paper, we aim to reduce such errors by incorporating the distances between pixels and keypoints into our objective. To this end, we develop a simple yet effective differentiable proxy voting loss (DPVL) which mimics the hypothesis selection in the voting procedure. By exploiting our voting loss, we are able to train our network in an end-to-end manner. Experiments on widely used datasets, i.e., LINEMOD and Occlusion LINEMOD, manifest that our DPVL improves pose estimation performance significantly and speeds up the training convergence.

Motivation & Objective

• To address the challenge of inaccurate 6DoF pose estimation under occlusions and textureless appearances in single-image RGB input.
• To reduce hypothesis deviation in keypoint voting caused by small direction vector errors in pixels far from keypoints.
• To develop a differentiable loss that mimics the voting procedure for end-to-end training, overcoming the non-differentiability of traditional voting methods like RANSAC.
• To improve pose estimation robustness and training convergence speed without retraining on occluded data.
• To enable accurate keypoint localization via proxy hypotheses that account for both direction vector accuracy and pixel-to-keypoint distance.

Proposed method

• DPVL introduces a proxy hypothesis for each pixel using the foot of the perpendicular from the ground-truth keypoint to the line defined by the pixel and its predicted direction vector.
• The loss function minimizes the distance between each proxy hypothesis and its corresponding ground-truth keypoint, integrating both direction vector accuracy and spatial distance.
• The computation of the foot of the perpendicular is differentiable, enabling backpropagation through the voting approximation during end-to-end training.
• The method combines the proxy voting loss with standard vector-field regression loss via a learnable weight λ, balancing both objectives.
• The network architecture and data pipeline are kept identical to PVNet for fair comparison, with only the loss function modified.
• Training convergence is accelerated, with the model reaching optimal performance in 100 epochs versus 200 for PVNet.

Experimental results

Research questions

• RQ1 How can we improve 6DoF object pose estimation under occlusion and textureless conditions using differentiable supervision?
• RQ2 Why do small direction vector errors lead to large hypothesis deviations in voting-based keypoint estimation?
• RQ3 Can we design a differentiable loss that approximates the voting process and accounts for pixel-to-keypoint distance?
• RQ4 Does incorporating spatial distance into the loss improve keypoint localization accuracy and training convergence?
• RQ5 Can a differentiable proxy voting loss outperform standard vector-field regression in pose estimation without retraining on occluded data?

Key findings

• The proposed DPVL achieves state-of-the-art performance on both LINEMOD and Occlusion LINEMOD datasets, outperforming PVNet and other SOTA methods.
• On the Occlusion LINEMOD dataset, the method achieves the best overall ADD(-S) score, demonstrating superior robustness to occlusion.
• The model converges significantly faster, reaching optimal performance in 100 training epochs compared to 200 for PVNet.
• The distribution of proxy hypotheses is more concentrated around ground-truth keypoints, indicating improved vector-field quality and voting consistency.
• Ablation studies confirm that λ = 1e-3 provides the best trade-off, as too large or too small values degrade performance.
• The method generalizes well, achieving strong results without fine-tuning on occluded data, unlike methods that require retraining on such data.