[Paper Review] Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image

Thanh-Toan Do, Ming Cai+2 more|arXiv (Cornell University)|Feb 28, 2018

Advanced Neural Network ApplicationsComputer Science22 references129 citations

TL;DR

Deep-6DPose jointly detects, segments, and regresses 6D poses of object instances from a single RGB image in an end-to-end framework, using a Lie algebra representation for rotations to enable direct pose regression without post-refinement.

ABSTRACT

Detecting objects and their 6D poses from only RGB images is an important task for many robotic applications. While deep learning methods have made significant progress in visual object detection and segmentation, the object pose estimation task is still challenging. In this paper, we introduce an end-toend deep learning framework, named Deep-6DPose, that jointly detects, segments, and most importantly recovers 6D poses of object instances from a single RGB image. In particular, we extend the recent state-of-the-art instance segmentation network Mask R-CNN with a novel pose estimation branch to directly regress 6D object poses without any post-refinements. Our key technical contribution is the decoupling of pose parameters into translation and rotation so that the rotation can be regressed via a Lie algebra representation. The resulting pose regression loss is differential and unconstrained, making the training tractable. The experiments on two standard pose benchmarking datasets show that our proposed approach compares favorably with the state-of-the-art RGB-based multi-stage pose estimation methods. Importantly, due to the end-to-end architecture, Deep-6DPose is considerably faster than competing multi-stage methods, offers an inference speed of 10 fps that is well suited for robotic applications.

Motivation & Objective

• Motivate end-to-end estimation of 6D object pose directly from RGB images without post-refinement.
• Leverage and extend Mask R-CNN with a dedicated pose regression head to recover 6D poses inside RoIs.
• Introduce a rotation representation via Lie algebra so(3) to enable unconstrained regression of rotations.
• Enable translation recovery using projected image cues together with predicted 2D bounding boxes.
• Demonstrate state-of-the-art or competitive performance on standard RGB-based pose benchmarks while achieving fast inference.

Proposed method

• Base on Mask R-CNN/Faster R-CNN backbone with Region Proposal Network to generate RoIs.
• Add a novel 6D pose head that regresses a 4D vector per RoI: first three components are so(3) rotation (Lie algebra), last component is z translation.
• Represent rotation using so(3) via Rodrigues mapping to obtain the rotation matrix.
• Recover full translation from the predicted z component and the bounding box via projective geometry (t_x and t_y from t_z and intrinsics).
• Train with a multi-task loss combining classification, box regression, mask segmentation, and pose regression losses.
• Pose branch is class-agnostic but can be extended to class-specific outputs.

Experimental results

Research questions

• RQ1 Can an end-to-end RGB-only network jointly detect, segment, and estimate 6D object poses without post-refinement?
• RQ2 Does representing rotation with Lie algebra so(3) facilitate stable, unconstrained regression in a CNN?
• RQ3 What is the impact of regressing only the z-translation component and recovering x/y from projection?
• RQ4 How does the end-to-end Deep-6DPose compare to state-of-the-art RGB-based pose methods on standard datasets in terms of accuracy and speed?

Key findings

• Deep-6DPose achieves competitive 2D detection/segmentation accuracy on LINEMOD and Tejani et al. datasets, with near-perfect scores at IoU 0.5 for detection and segmentation.
• Under 5cm/5° pose accuracy, Deep-6DPose outperforms Brachmann et al. and is competitive with BB8, while SSD-6D can be stronger due to synthetic training data, but Deep-6DPose offers end-to-end pose output without refinement.
• On LINEMOD, Deep-6DPose shows 2D-pose metric comparable to SSD-6D and better than Brachmann et al., and ADD metric performance surpasses BB8 by about 2.5%.
• On Tejani et al. dataset for multiple instances, Deep-6DPose attains near-perfect 2D detection/segmentation at IoU 0.5 and reasonable 5cm/5° and ADD scores on average, with some degradation on nearly symmetric objects.
• Inference speed is around 0.1s per image on a Titan X, making it several times faster than multi-stage methods and faster than BB8, while being competitive with SSD-6D.