[Paper Review] Real-time Scalable Dense Surfel Mapping

Kaixuan Wang, Fei Gao+1 more|arXiv (Cornell University)|Sep 10, 2019

Robotics and Sensor-Based Localization23 references4 citations

TL;DR

This paper proposes a real-time, CPU-only dense surfel mapping system that achieves global consistency in large-scale environments by using superpixel-based surfels and pose graph-driven fusion. By organizing surfels according to the SLAM pose graph, the method maintains O(1) fusion time and enables fast map deformation for loop closure correction, enabling scalable, memory-efficient reconstruction from RGB-D, stereo, or monocular depth inputs without GPU acceleration.

ABSTRACT

In this paper, we propose a novel dense surfel mapping system that scales well in different environments with only CPU computation. Using a sparse SLAM system to estimate camera poses, the proposed mapping system can fuse intensity images and depth images into a globally consistent model. The system is carefully designed so that it can build from room-scale environments to urban-scale environments using depth images from RGB-D cameras, stereo cameras or even a monocular camera. First, superpixels extracted from both intensity and depth images are used to model surfels in the system. superpixel-based surfels make our method both run-time efficient and memory efficient. Second, surfels are further organized according to the pose graph of the SLAM system to achieve $O(1)$ fusion time regardless of the scale of reconstructed models. Third, a fast map deformation using the optimized pose graph enables the map to achieve global consistency in real-time. The proposed surfel mapping system is compared with other state-of-the-art methods on synthetic datasets. The performances of urban-scale and room-scale reconstruction are demonstrated using the KITTI dataset and autonomous aggressive flights, respectively. The code is available for the benefit of the community.

Motivation & Objective

• To enable real-time, globally consistent dense 3D reconstruction in large-scale environments using only CPU computation.
• To address the scalability and memory efficiency challenges of dense surfel mapping in room-scale to urban-scale environments.
• To support fusion of low-quality depth maps from stereo and monocular cameras without GPU acceleration.
• To maintain real-time performance and global consistency through efficient surfel organization and pose graph-based map deformation.
• To provide a robust, general-purpose mapping system usable across diverse robotic platforms and sensor types.

Proposed method

• Surfels are initialized using outlier-robust superpixels extracted from intensity and depth images, reducing memory usage and improving robustness to noisy depth.
• Surfels are organized based on the keyframes in which they were last observed, enabling localized fusion and O(1) fusion time regardless of model scale.
• The system leverages the optimized pose graph from a sparse SLAM system (e.g., ORB-SLAM2) to identify locally consistent surfel clusters for efficient fusion.
• Map deformation is performed in real-time using the optimized pose graph to correct drift and ensure global consistency after loop closures.
• The method supports fusion of depth maps from RGB-D cameras, stereo cameras, and monocular depth prediction networks, enabling flexibility across sensor types.
• Runtime efficiency is achieved by offloading most computation to superpixel extraction and surfel initialization, while surfel fusion itself takes less than 6 ms per frame.

Experimental results

Research questions

• RQ1 Can a dense surfel mapping system achieve real-time performance and global consistency in urban-scale environments using only CPU computation?
• RQ2 How can surfel-based mapping maintain O(1) fusion time across models of varying scale without GPU acceleration?
• RQ3 Can superpixel-based surfel representation improve robustness to low-quality depth maps from stereo or monocular cameras?
• RQ4 How effectively can pose graph-based surfel organization and deformation correct map drift in large-scale reconstructions?
• RQ5 Can the system support seamless reconstruction across diverse sensor inputs (RGB-D, stereo, monocular) while maintaining memory and runtime efficiency?

Key findings

• The system achieves 10 Hz real-time performance on the KITTI odometry sequence 00 using stereo depth maps and ORB-SLAM2, with less than 6 ms per-frame surfel fusion time.
• Memory usage grows only with environment scale, not runtime, and remains nearly constant during loop closures due to surfel reuse, as demonstrated in the KITTI sequence between frames 3000 and 4000.
• The method successfully reconstructs urban-scale scenes using monocular depth prediction, making it the first online dense fusion system to achieve scale-consistent reconstruction from monocular inputs without GPU.
• Map drift is effectively corrected in real-time after loop closures, as shown in the deformation of a street corner where misaligned structures (e.g., road and car) are realigned post-deformation.
• Compared to CHISEL, the proposed method enables accurate reconstruction of fine obstacles during repeated visits, making the map usable for autonomous aggressive flights.
• The system outperforms existing methods in scalability and memory efficiency, maintaining real-time performance on CPU while supporting reconstruction from low-quality depth inputs.