[Paper Review] 6D Object Pose Estimation Based on 2D Bounding Box

Jin Liu, Sheng He|arXiv (Cornell University)|Jan 27, 2019

Image and Object Detection Techniques26 references4 citations

TL;DR

This paper proposes a novel 6D object pose estimation method that leverages a 2D object detection bounding box to predict 3D rotation via a unit quaternion-regression network (Q-Net) and 3D translation using a new Bounding Box Equation. The approach achieves state-of-the-art performance on the LineMod dataset without requiring depth data or 3D point clouds, outperforming prior methods in both rotation and pose accuracy metrics.

ABSTRACT

In this paper, we present a simple but powerful method to tackle the problem of estimating the 6D pose of objects from a single RGB image. Our system trains a novel convolutional neural network to regress the unit quaternion, which represents the 3D rotation, from the partial image inside the bounding box returned by 2D detection systems. Then we propose an algorithm we call Bounding Box Equation to efficiently and accurately obtain the 3D translation, using 3D rotation and 2D bounding box. Considering that the quadratic sum of the quaternion's four elements equals to one, we add a normalization layer to keep the network's output on the unit sphere and put forward a special loss function for unit quaternion regression. We evaluate our method on the LineMod dataset and experiment shows that our approach outperforms base-line and some state of the art methods.

Motivation & Objective

• To address the challenge of 6D object pose estimation from single RGB images without depth sensors or 3D supervision.
• To enable accurate pose estimation for both textured and texture-less objects using only 2D detection outputs.
• To develop a lightweight, efficient framework compatible with existing 2D object detectors for real-time deployment.
• To improve unit quaternion regression via a specialized normalization layer and Dot Product Loss function.
• To introduce a novel Bounding Box Equation for accurate 3D translation estimation using 2D bounding box and 3D rotation.

Proposed method

• A novel CNN, Q-Net, is trained to regress the unit quaternion representing 3D rotation from the cropped image inside a 2D bounding box.
• A normalization layer ensures the network output remains on the unit sphere, maintaining the quaternion constraint.
• A custom loss function, Dot Product Loss, is designed for unit vector regression to improve training stability and accuracy.
• The Bounding Box Equation computes 3D translation using the estimated 3D rotation and 2D bounding box coordinates, leveraging 3D model geometry (e.g., point cloud or 8 corners).
• The method operates end-to-end: 2D detection → 3D rotation prediction → 3D translation estimation → full 6D pose.
• The approach is evaluated on LineMod with full 3D point clouds and on daily objects using only 8 corners for translation estimation.

Experimental results

Research questions

• RQ1 Can 6D object pose be accurately estimated from a single RGB image using only a 2D bounding box and no depth data?
• RQ2 How can unit quaternion regression be improved for 3D rotation estimation in a deep learning framework?
• RQ3 Can a closed-form equation efficiently and accurately compute 3D translation from 2D bounding box and 3D rotation?
• RQ4 Does the proposed method generalize to both textured and texture-less objects in real-world scenarios?
• RQ5 How does the method compare to state-of-the-art approaches in terms of rotation and full pose accuracy?

Key findings

• On the LineMod dataset, the method achieves an average Euler angle error of 5.21°, outperforming prior methods such as [16] (11.48°) and [33] (14.53°).
• The method achieves a 58.07% success rate under the 5cm/5° threshold on LineMod, surpassing the 40.60% of [34] and demonstrating strong generalization.
• For the challenging eggbox and glue objects, the method achieves 60.0% and 59.6% success rates, respectively, outperforming [34]’s 57.1% and 23.6%.
• The full pipeline runs in 19ms on a GTX 1080 GPU (16ms for 2D detection, 2ms for rotation, 1ms for translation), indicating real-time feasibility.
• The method generalizes well to daily objects (e.g., cup, radio, book) without requiring 3D models, achieving visually accurate pose predictions.
• The ablation study confirms that the Dot Product Loss and normalization layer significantly improve unit quaternion regression performance.