[Paper Review] GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM

Gan‐Lin Zhang, Erik Sandström+4 more|arXiv (Cornell University)|Mar 28, 2024

3D Surveying and Cultural Heritage6 citations

TL;DR

GlORIE-SLAM presents an RGB-only dense SLAM system using a deformable neural point cloud with online loop closure and global BA, augmented by a DSPO layer that fuses monocular depth priors into pose-depth optimization.

ABSTRACT

Recent advancements in RGB-only dense Simultaneous Localization and Mapping (SLAM) have predominantly utilized grid-based neural implicit encodings and/or struggle to efficiently realize global map and pose consistency. To this end, we propose an efficient RGB-only dense SLAM system using a flexible neural point cloud scene representation that adapts to keyframe poses and depth updates, without needing costly backpropagation. Another critical challenge of RGB-only SLAM is the lack of geometric priors. To alleviate this issue, with the aid of a monocular depth estimator, we introduce a novel DSPO layer for bundle adjustment which optimizes the pose and depth of keyframes along with the scale of the monocular depth. Finally, our system benefits from loop closure and online global bundle adjustment and performs either better or competitive to existing dense neural RGB SLAM methods in tracking, mapping and rendering accuracy on the Replica, TUM-RGBD and ScanNet datasets. The source code is available at https://github.com/zhangganlin/GlOIRE-SLAM

Motivation & Objective

• Motivate RGB-only dense SLAM without costly backpropagation for map updates.
• Propose a deformable neural point cloud map enabling online loop closure and global BA.
• Introduce DSPO (Disparity, Scale and Pose Optimization) to fuse monocular depth priors into BA.
• Leverage monocular depth priors to improve scale, depth accuracy, and reconstruction quality.
• Demonstrate competitive or superior rendering, reconstruction, and tracking performance on standard datasets.

Proposed method

• Represent the scene with a deformable neural point cloud carrying geometric and color features per point.
• Add neural points along rays for keyframes with depth-guided sampling and a depth-aware dynamic radius.
• Render depth and color via depth-guided volume rendering using a fixed occupancy MLP and a color MLP with view direction conditioning.
• Tracking uses frame-to-frame optical-flow with a DBA-based local BA and a DSPO layer that jointly optimizes disparity, scale, and pose using monocular depth priors.
• Proxy depth maps are constructed by fusing monocular depth priors with multi-view consistency to stabilize rendering and mapping.
• Mapping updates deform the neural point cloud to reflect refined camera poses without retraining features.
• Implement loop closure and online global BA via a factor-graph approach, with normalization steps for numerical stability.
• Two-stage optimization for DSPO: first refine geometry with DBA; then jointly optimize high-error disparities along with scale and shift guided by monocular depth priors.
• Training-free mapping: maintain deformable point cloud so that global pose changes are reflected without backpropagation to map encodings.

Experimental results

Research questions

• RQ1 Can an RGB-only SLAM system achieve globally consistent maps and accurate tracking without backpropagating neural grid features?
• RQ2 Does a deformable neural point cloud with online loop closure and global BA provide improved rendering and reconstruction over RGB-only grid-based methods?
• RQ3 How can monocular depth priors be effectively integrated into a DSPO layer to stabilize scale and depth in RGB-only SLAM?
• RQ4 What is the impact of online loop closure and global BA on trajectory accuracy and map quality across Replica, TUM-RGBD, and ScanNet?
• RQ5 How does the proposed method perform in rendering, reconstruction, and tracking compared to contemporary RGB and RGB-D dense SLAM baselines?

Key findings

• The method achieves higher rendering accuracy (PSNR and SSIM) and competitive LPIPS scores compared to RGB baselines on Replica/ScanNet.
• GlORIE-SLAM attains best or competitive ATE RMSE among RGB methods, indicating strong tracking accuracy.
• The deformable neural point cloud enables online map deformation during loop closure and global BA, avoiding full map re-creation.
• DSPO layer improves depth and pose estimates by coupling monocular depth priors with disparity and scale refinement.
• On ScanNet, GlORIE-SLAM outperforms many RGB-D methods in rendering metrics, and shows robust tracking accuracy across datasets.