[Paper Review] StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS

Kai Chen, Stephen James+4 more|arXiv (Cornell University)|Nov 3, 2022

Robotics and Sensor-Based Localization4 citations

TL;DR

StereoPose proposes a stereo image-based framework for category-level 6D pose estimation of transparent objects, leveraging a novel back-view NOCS map to reduce ambiguity from image content aliasing and self-occlusion. By combining parallax-aware stereo feature fusion, epipolar geometry loss, and a three-stage pipeline (size estimation, initial pose, refinement), it achieves state-of-the-art performance on the TOD dataset without relying on depth maps.

ABSTRACT

Most existing methods for category-level pose estimation rely on object point clouds. However, when considering transparent objects, depth cameras are usually not able to capture meaningful data, resulting in point clouds with severe artifacts. Without a high-quality point cloud, existing methods are not applicable to challenging transparent objects. To tackle this problem, we present StereoPose, a novel stereo image framework for category-level object pose estimation, ideally suited for transparent objects. For a robust estimation from pure stereo images, we develop a pipeline that decouples category-level pose estimation into object size estimation, initial pose estimation, and pose refinement. StereoPose then estimates object pose based on representation in the normalized object coordinate space~(NOCS). To address the issue of image content aliasing, we further define a back-view NOCS map for the transparent object. The back-view NOCS aims to reduce the network learning ambiguity caused by content aliasing, and leverage informative cues on the back of the transparent object for more accurate pose estimation. To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions. Extensive experiments on the public TOD dataset demonstrate the superiority of the proposed StereoPose framework for category-level 6D transparent object pose estimation.

Motivation & Objective

• To address the challenge of category-level 6D pose estimation for transparent objects, which suffer from poor depth map quality due to specular and reflective materials.
• To overcome the limitations of existing point-cloud-based methods that fail on transparent objects due to severe artifacts in depth data.
• To reduce learning ambiguity caused by image content aliasing, where object structures on the back appear in front-view images.
• To exploit informative cues on the back of transparent objects (e.g., handles, rims) that are typically ignored in front-view-only representations.
• To develop a stereo image-based framework that achieves robust and accurate pose estimation without relying on depth maps or high-quality point clouds.

Proposed method

• The framework decouples category-level pose estimation into three stages: object size estimation, initial pose estimation, and pose refinement.
• It introduces a back-view NOCS map to model dense correspondences for the back side of transparent objects, reducing ambiguity from aliased image content.
• A parallax attention module fuses stereo features using geometric parallax as a guidance signal, enhancing cross-view feature consistency.
• An epipolar geometry loss enforces consistency between left and right view NOCS predictions by aligning corresponding points via epipolar constraints.
• The network jointly predicts both front-view and back-view NOCS maps from stereo RGB images, improving overall correspondence accuracy.
• The method avoids explicit depth map usage and instead leverages stereo geometry and NOCS representation for robust 6D pose estimation.

Experimental results

Research questions

• RQ1 Can a stereo image-based framework achieve accurate category-level 6D pose estimation for transparent objects without relying on depth maps?
• RQ2 How does introducing a back-view NOCS map reduce learning ambiguity caused by image content aliasing in transparent objects?
• RQ3 To what extent does parallax-aware stereo feature fusion improve pose estimation accuracy compared to standard stereo fusion?
• RQ4 How effective is the epipolar geometry loss in improving stereo-view consistency of NOCS predictions?
• RQ5 What is the contribution of back-side structural cues (e.g., handles, rims) to pose accuracy in transparent object estimation?

Key findings

• The proposed StereoPose framework achieves 97.9% 3D IoU@25 on the TOD dataset, significantly outperforming baseline methods.
• Removing the back-view NOCS map causes a 9.6% drop in 3D IoU@25, demonstrating its critical role in handling back-side structure cues.
• The parallax attention module improves 3D IoU@25 by 4.2% compared to a baseline without stereo fusion.
• The epipolar geometry loss increases 3D IoU@25 by 3.5% when removed, showing its effectiveness in enforcing stereo consistency.
• The full model achieves 77.4% 3D IoU@50 and 38.2% 10°10cm accuracy, outperforming ablation variants across all metrics.
• Qualitative results confirm that the back-view NOCS map enables accurate localization of handles and other back-side features, reducing rotation errors in challenging configurations.