[Paper Review] Probabilistic Combination of Noisy Points and Planes for RGB-D Odometry

Pedro F. Proença, Yang Gao|arXiv (Cornell University)|May 18, 2017

Robotics and Sensor-Based Localization4 citations

TL;DR

This paper proposes a probabilistic visual odometry method for RGB-D cameras that combines 3D points and planar primitives by modeling depth sensor uncertainty and propagating it through geometric feature extraction to optimize camera pose via uncertainty-weighted residuals. The approach significantly improves robustness in low-texture environments—reducing relative translational error by 30% on the fr3/structure_notexture_far sequence—compared to point-only methods.

ABSTRACT

This work proposes a visual odometry method that combines points and plane primitives, extracted from a noisy depth camera. Depth measurement uncertainty is modelled and propagated through the extraction of geometric primitives to the frame-to-frame motion estimation, where pose is optimized by weighting the residuals of 3D point and planes matches, according to their uncertainties. Results on an RGB-D dataset show that the combination of points and planes, through the proposed method, is able to perform well in poorly textured environments, where point-based odometry is bound to fail.

Motivation & Objective

• To address the failure of point-based visual odometry in low-texture or blur-prone indoor environments where feature points are sparse.
• To improve pose estimation accuracy by modeling and propagating depth measurement uncertainty from raw sensor data through geometric primitive extraction.
• To combine 3D points and planar primitives in a unified probabilistic framework that weights residuals according to their uncertainty.
• To demonstrate that plane-based features, when properly weighted by uncertainty, enhance robustness in structured-light RGB-D odometry.
• To provide a principled alternative to Euclidean distance minimization in ICP-style alignment by using uncertainty-informed weighting.

Proposed method

• Uses SURF features for 2D point detection in RGB images, followed by back-projection into 3D using calibrated depth values and a known depth uncertainty model (σZ = 1.425×10⁻⁶Z² mm) for structured-light cameras.
• Applies a weighted least squares plane fitting method that incorporates depth uncertainty into the estimation of plane parameters (normal vector and distance from origin).
• Proposes a novel plane-to-plane distance metric as an alternative to standard point-to-plane distances for pose optimization.
• Performs iterative reweighted least squares optimization of camera pose, where residuals from both points and planes are scaled by their respective uncertainties to minimize the overall error.
• Uses a tunable parameter α to balance the influence of plane residuals relative to point residuals in the cost function.
• Derives uncertainty propagation from depth measurements through 3D point and plane estimation using first-order error propagation and Jacobian matrices.

Experimental results

Research questions

• RQ1 Can combining 3D points and planar primitives in a probabilistic framework improve visual odometry performance in low-texture environments?
• RQ2 How does modeling depth sensor uncertainty affect pose estimation accuracy in RGB-D odometry?
• RQ3 What is the optimal balance between point and plane features in terms of uncertainty-weighted residuals?
• RQ4 Does uncertainty-aware weighting of geometric features reduce drift and improve robustness in challenging sequences?
• RQ5 Can a novel plane-to-plane distance metric improve the stability and accuracy of pose estimation compared to traditional point-to-plane methods?

Key findings

• The proposed uncertainty-weighted point-based odometry reduced relative translational error from 43 mm/s to 38 mm/s on the fr1/desk sequence, demonstrating the benefit of uncertainty modeling.
• On the fr1/360 sequence, which features high motion blur and low feature density, uncertainty-weighted point odometry reduced error from 109 mm/s to 86 mm/s.
• The combination of points and planes with α = 10 reduced translational error to 26 mm/s on the fr1/desk sequence, outperforming the best published monocular method (26 mm/s vs. 43 mm/s).
• On the fr3/structure_notexture_far sequence, where point-based odometry failed, the combined method achieved 50 mm/s error, a 30% improvement over the state-of-the-art 43 mm/s.
• The method achieved a 17% improvement in accuracy on the fr3/cabinet sequence (80 mm/s to 73 mm/s) by leveraging plane features in low-texture conditions.
• The optimal α value of 10 was found to balance point and plane contributions, with higher α favoring planes in low-texture scenes and lower α favoring points in textured scenes.