[Paper Review] Point-SLAM: Dense Neural Point Cloud-based SLAM

Erik Sandström, Yue Li+2 more|arXiv (Cornell University)|Apr 9, 2023

Robotics and Sensor-Based Localization18 citations

TL;DR

Point-SLAM presents a dense neural SLAM system that anchors learnable point features in a data-driven, adaptive neural point cloud, enabling real-time tracking and mapping from monocular RGBD inputs with improved efficiency and detail in high-information regions.

ABSTRACT

We propose a dense neural simultaneous localization and mapping (SLAM) approach for monocular RGBD input which anchors the features of a neural scene representation in a point cloud that is iteratively generated in an input-dependent data-driven manner. We demonstrate that both tracking and mapping can be performed with the same point-based neural scene representation by minimizing an RGBD-based re-rendering loss. In contrast to recent dense neural SLAM methods which anchor the scene features in a sparse grid, our point-based approach allows dynamically adapting the anchor point density to the information density of the input. This strategy reduces runtime and memory usage in regions with fewer details and dedicates higher point density to resolve fine details. Our approach performs either better or competitive to existing dense neural RGBD SLAM methods in tracking, mapping and rendering accuracy on the Replica, TUM-RGBD and ScanNet datasets. The source code is available at https://github.com/eriksandstroem/Point-SLAM.

Motivation & Objective

• Motivate dense SLAM with a neural point-based scene representation that adapts to input information density.
• Develop a dynamic, input-driven point cloud that serves as both the map and the tracker.
• Render depth and color via volume rendering from neural point features to optimize a re-rendering loss.
• Demonstrate real-time capable tracking and mapping with improved detail where needed while reducing computation elsewhere.

Proposed method

• Introduce a neural point cloud P with points p_i and features f_i^g (geometry) and f_i^c (color).
• Grow the point cloud iteratively as the scene is explored, adding points only when no nearby neighbors exist within a radius r.
• Implement dynamic point density by varying the search radius r as a function of image gradient to concentrate points in textured regions.
• Render depth and color by sampling along rays and decoding occupancies o_i and colors c_i using geometry decoder h and color decoder g_ξ with neural features P^g and P^c.
• Minimize an RGBD re-rendering loss for mapping (depth and color) and tracking (pose) to optimize the map and camera trajectory.
• Use exposure compensation via a learned per-image latent vector and an exposure MLP G_φ to handle scene illumination changes.

Experimental results

Research questions

• RQ1 Can point-based neural scene representations support both tracking and mapping in a real-time dense RGBD SLAM system?
• RQ2 Does input-adaptive (data-driven) point anchoring improve rendering, mapping, and tracking accuracy compared to grid-based neural SLAM methods?
• RQ3 What are the effects of dynamic point density on memory, runtime, and reconstruction quality across indoor datasets?
• RQ4 How does Point-SLAM perform on standard benchmarks (Replica, TUM-RGBD, ScanNet) relative to NICE-SLAM, Vox-Fusion, and ESLAM?

Key findings

• Point-SLAM achieves state-of-the-art or competitive performance in tracking, rendering, and mapping on Replica, TUM-RGBD, and ScanNet.
• Adaptive, data-driven point anchoring improves rendering fidelity and reconstruction in high-frequency detail regions (e.g., textured areas) while reducing memory and compute in textureless regions.
• Compared to dense grid-based methods, dynamic point density provides better memory-accuracy trade-offs without sacrificing tracking accuracy.
• On Replica, Point-SLAM attains lower tracking RMSE (ATE RMSE) on average than NICE-SLAM, Vox-Fusion, and ESLAM in the reported runs.
• Table 1 shows Point-SLAM achieving superior or competitive tracking performance (AVE RMSE across Rm0–Off4) relative to baselines on Replica.