[Paper Review] FMapping: Factorized Efficient Neural Field Mapping for Real-Time Dense RGB SLAM

Tongyan Hua, Haotian Bai+2 more|arXiv (Cornell University)|Jun 1, 2023

Robotics and Sensor-Based Localization4 citations

TL;DR

FMapping proposes a factorized neural field mapping framework for real-time dense RGB SLAM that reduces mapping uncertainty and improves efficiency without depth sensors. By decomposing scene representation into lower-dimensional latent spaces and using a sliding window sampler for robust initialization, it achieves high-fidelity, real-time colorized point cloud reconstruction with 20× fewer parameters than iMAP and 1000× fewer than NICE-SLAM, running at ~2 seconds per 20 frames using only PyTorch.

ABSTRACT

In this paper, we introduce FMapping, an efficient neural field mapping framework that facilitates the continuous estimation of a colorized point cloud map in real-time dense RGB SLAM. To achieve this challenging goal without depth, a hurdle is how to improve efficiency and reduce the mapping uncertainty of the RGB SLAM system. To this end, we first build up a theoretical analysis by decomposing the SLAM system into tracking and mapping parts, and the mapping uncertainty is explicitly defined within the frame of neural representations. Based on the analysis, we then propose an effective factorization scheme for scene representation and introduce a sliding window strategy to reduce the uncertainty for scene reconstruction. Specifically, we leverage the factorized neural field to decompose uncertainty into a lower-dimensional space, which enhances robustness to noise and improves training efficiency. We then propose the sliding window sampler to reduce uncertainty by incorporating coherent geometric cues from observed frames during map initialization to enhance convergence. Our factorized neural mapping approach enjoys some advantages, such as low memory consumption, more efficient computation, and fast convergence during map initialization. Experiments on two benchmark datasets show that our method can update the map of high-fidelity colorized point clouds around 2 seconds in real time while requiring no customized CUDA kernels. Additionally, it utilizes x20 fewer parameters than the most concise neural implicit mapping of prior methods for SLAM, e.g., iMAP [ 31] and around x1000 fewer parameters than the state-of-the-art approach, e.g., NICE-SLAM [ 42]. For more details, please refer to our project homepage: https://vlis2022.github.io/fmap/.

Motivation & Objective

• To address the challenge of real-time dense RGB SLAM without depth sensors, where mapping uncertainty and computational cost are major bottlenecks.
• To reduce memory and computation overhead in neural implicit mapping for dense SLAM, especially in resource-constrained environments.
• To improve robustness and convergence during map initialization by leveraging coherent geometric cues from multiple frames.
• To establish a theoretical framework for analyzing and decomposing mapping uncertainty within neural representation-based SLAM.
• To achieve high-fidelity, continuous reconstruction of colorized point clouds in real time using only RGB inputs and no depth supervision.

Proposed method

• The method introduces a factorized neural field representation that decomposes scene uncertainty into a lower-dimensional latent space, improving noise robustness and training efficiency.
• A sliding window sampler is proposed to integrate geometric cues from temporally coherent frames during map initialization, enhancing convergence and reducing uncertainty.
• The framework formulates mapping as maximizing the posterior probability conditioned on learned latent features from incremental RGB observations.
• The factorized representation reduces parameter count by up to ×1000 compared to NICE-SLAM and ×20 compared to iMAP, enabling efficient inference.
• The system avoids custom CUDA kernels and runs in real time using standard PyTorch, achieving 2.01 seconds per 20 frames on benchmark sequences.
• The method is evaluated under full RGB-only settings, with no depth supervision, relying solely on color gradients for pose and geometry optimization.

Experimental results

Research questions

• RQ1 How can mapping uncertainty in depth-free dense RGB SLAM be theoretically modeled and decomposed within a neural representation framework?
• RQ2 What factorization strategy in neural field representation can effectively reduce parameter count and improve training efficiency without sacrificing reconstruction fidelity?
• RQ3 How can a sliding window sampling strategy enhance map initialization and reduce uncertainty during early reconstruction phases?
• RQ4 To what extent can a factorized neural field achieve real-time performance with minimal memory and computation, especially compared to SOTA methods like iMAP and NICE-SLAM?
• RQ5 Can high-fidelity, continuous colorized point cloud reconstruction be achieved in real time using only RGB inputs and no depth supervision?

Key findings

• FMapping achieves real-time mapping with a processing time of 2.01 seconds per 20 frames, enabling continuous reconstruction of high-fidelity colorized point clouds.
• The method uses 0.01 million parameters, representing a ×20 reduction compared to iMAP(-) and a ×1000 reduction compared to NICE-SLAM, significantly lowering memory footprint.
• FMapping achieves a mean completion ratio of 79.8% across Replica datasets, outperforming iMAP(-) by nearly 2× in completion ratio and showing superior reconstruction fidelity.
• The model requires only 80.64×10³ FLOPs per 3D point, which is 1/5 of iMAP’s computation cost, demonstrating superior computational efficiency.
• The method maintains robust performance without custom CUDA kernels, achieving real-time inference using only standard PyTorch, unlike DIM-SLAM which requires tailored acceleration.
• In qualitative evaluation, FMapping produces consistently complete and accurate reconstructions throughout the entire SLAM process, while iMAP(-) fails to reconstruct geometry in long sequences.