[Paper Review] L6DNet: Light 6 DoF Network for Robust and Precise Object Pose Estimation with Small Datasets

Mathieu Gonzalez, Amine Kacete+2 more|arXiv (Cornell University)|Feb 3, 2020

Robot Manipulation and Learning44 references16 citations

TL;DR

L6DNet proposes a two-stage hybrid method for 6 DoF object pose estimation from a single RGB-D image, using a cascade of CNNs to predict 3D keypoint locations in camera coordinates, followed by geometric registration to compute the final pose. It achieves state-of-the-art accuracy on the LineMod dataset and enables real-time visual servoing with high precision and robustness, especially with small datasets and textureless objects.

ABSTRACT

Estimating the 3D pose of an object is a challenging task that can be considered within augmented reality or robotic applications. In this paper, we propose a novel approach to perform 6 DoF object pose estimation from a single RGB-D image. We adopt a hybrid pipeline in two stages: data-driven and geometric respectively. The data-driven step consists of a classification CNN to estimate the object 2D location in the image from local patches, followed by a regression CNN trained to predict the 3D location of a set of keypoints in the camera coordinate system. To extract the pose information, the geometric step consists in aligning the 3D points in the camera coordinate system with the corresponding 3D points in world coordinate system by minimizing a registration error, thus computing the pose. Our experiments on the standard dataset LineMod show that our approach is more robust and accurate than state-of-the-art methods. The approach is also validated to achieve a 6 DoF positioning task by visual servoing.

Motivation & Objective

• Address the challenge of accurate and robust 6 DoF object pose estimation in complex scenes with occlusions, textureless objects, and varying lighting.
• Overcome limitations of end-to-end deep learning methods by decoupling keypoint prediction from pose computation using geometric registration.
• Enable high-accuracy pose estimation with small training datasets, reducing reliance on large-scale annotated data.
• Design a method that is robust to image variations and suitable for real-time robotic applications like visual servoing.

Proposed method

• Use a two-stage pipeline: a data-driven stage for 3D keypoint prediction and a geometric stage for pose computation via 3D-3D registration.
• First, a classification CNN processes local image patches to estimate 2D object locations, enabling selective keypoint regression.
• Then, a regression CNN predicts 3D coordinates of sparse keypoints in the camera coordinate system from RGB-D patches.
• Apply a voting mechanism across multiple patches to aggregate multiple 3D keypoint hypotheses into a single, robust estimate.
• Compute the final 6 DoF pose by minimizing a 3D-3D registration error between predicted camera-space points and known 3D world coordinates of object keypoints.
• Use a lightweight architecture with small receptive fields and low-stride patch extraction to maintain speed and accuracy.

Experimental results

Research questions

• RQ1 Can a hybrid learning-geometric approach outperform end-to-end deep learning methods in 6 DoF pose estimation under small dataset constraints?
• RQ2 Does predicting 3D keypoints in camera space and solving 3D-3D registration yield better accuracy than 2D-3D correspondence or direct pose regression?
• RQ3 How effective is the patch-based voting strategy in improving robustness and accuracy for textureless or occluded objects?
• RQ4 To what extent does the inclusion of depth data improve keypoint regression and pose estimation performance compared to RGB-only input?
• RQ5 Can the method achieve real-time performance and stability in a visual servoing task for robotic manipulation?

Key findings

• L6DNet achieves a mean ADD-S accuracy of 96.9% on the LineMod dataset using RGB-only input and 96.7% with RGB-D, demonstrating strong performance even with limited data.
• For the driller object, the method achieves 98.7% ADD-S accuracy with a patch size of 32×32 and a stride of 4 pixels, with inference time under 525 ms.
• The voting mechanism significantly improves accuracy: using a single 80×80 patch yields only 44.2% ADD-S for the ape, compared to 78.2% with multiple patches.
• The method achieves high visual servoing accuracy, with final error ∆r = (1.2mm, -1.6mm, -0.4mm, -0.09°, -0.18°, 0.07°) after 120 iterations, indicating stable and precise camera control.
• Inference time increases with patch size, peaking at 634.9 ms for 80×80 patches, but accuracy peaks at 64×64 before declining, indicating an optimal patch size trade-off.
• Depth input improves performance for some objects (e.g., cam, drill) but is less critical for others; RGB-only patches can be more discriminative for certain textures.