[Paper Review] How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey

Fabio Tosi, Youmin Zhang+5 more|arXiv (Cornell University)|Feb 20, 2024

Robotics and Sensor-Based Localization16 citations

TL;DR

This survey reviews SLAM progress through radiance-field inspired methods, focusing on Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), and catalogs 80 recent systems to map strengths, limitations, and future challenges.

ABSTRACT

Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges.

Motivation & Objective

• Survey the latest SLAM progress driven by radiance-field representations (NeRF and 3DGS).
• Categorize and analyze 80 SLAM systems published in the last three years.
• Identify strengths, limitations, and future research directions in radiance-field–based SLAM.
• Provide a reference framework to guide researchers and practitioners in this evolving field.

Proposed method

• Explain radiance-field theory and representations (implicit, explicit, hybrid).
• Describe NeRF and 3DGS foundations and how they are adapted for SLAM.
• Review datasets and benchmarks used in radiance-field SLAM evaluations.
• Offer a taxonomy of NeRF/3DGS–inspired SLAM methods and compare design choices (tracking vs mapping, sub-maps, dynamics, priors).
• Summarize evaluation metrics across mapping, tracking, view synthesis, and semantics.

Experimental results

Research questions

• RQ1 What are the key radiance-field representations enabling SLAM (implicit NeRF, explicit grids, 3D Gaussian splats) and how do they compare?
• RQ2 How have NeRFs and 3D Gaussian Splatting been integrated into SLAM pipelines (frame-to-frame vs frame-to-model) and what are the resulting trade-offs?
• RQ3 What datasets, benchmarks, and metrics are used to evaluate these radiance-field–based SLAM systems?
• RQ4 What are the current limitations and open challenges in NeRF/3DGS–driven SLAM, and what future directions are suggested?
• RQ5 How do these approaches perform across dynamic environments, scalability, and real-time requirements?

Key findings

• Radiance-field–based SLAM offers continuous surface modeling and potential for denser, more compact maps.
• NeRF and 3DGS provide distinct rendering and geometry representations with different trade-offs in speed, memory, and handling of empty space.
• The survey covers 80 SLAM systems published recently, highlighting rapid progress and diverse design strategies (sub-maps, dynamics handling, priors).
• Evaluations rely on standard SLAM metrics for mapping accuracy, tracking accuracy (ATE), and view synthesis quality (PSNR, SSIM, LPIPS).
• There is a clear shift in 2021–2024 toward radiance-field inspired SLAM, with increasing interest and tool availability (datasets, benchmarks, code).
• The survey frames a structured taxonomy to organize NeRF/3DGS–driven SLAM methods and points to practical limitations and research gaps.