[Paper Review] ActiveRMAP: Radiance Field for Active Mapping And Planning

Huangying Zhan, Jiyang Zheng+3 more|arXiv (Cornell University)|Nov 23, 2022

Advanced Vision and Imaging20 citations

TL;DR

ActiveRMAP presents an online RGB-only active vision framework that jointly maps and plans using a hybrid implicit-explicit radiance field (DVGO) without pre-training, achieving competitive rendering and reconstruction results compared with offline methods and outperforming some NeRF-based active approaches.

ABSTRACT

A high-quality 3D reconstruction of a scene from a collection of 2D images can be achieved through offline/online mapping methods. In this paper, we explore active mapping from the perspective of implicit representations, which have recently produced compelling results in a variety of applications. One of the most popular implicit representations - Neural Radiance Field (NeRF), first demonstrated photorealistic rendering results using multi-layer perceptrons, with promising offline 3D reconstruction as a by-product of the radiance field. More recently, researchers also applied this implicit representation for online reconstruction and localization (i.e. implicit SLAM systems). However, the study on using implicit representation for active vision tasks is still very limited. In this paper, we are particularly interested in applying the neural radiance field for active mapping and planning problems, which are closely coupled tasks in an active system. We, for the first time, present an RGB-only active vision framework using radiance field representation for active 3D reconstruction and planning in an online manner. Specifically, we formulate this joint task as an iterative dual-stage optimization problem, where we alternatively optimize for the radiance field representation and path planning. Experimental results suggest that the proposed method achieves competitive results compared to other offline methods and outperforms active reconstruction methods using NeRFs.

Motivation & Objective

• Motivate and enable online active mapping and planning using implicit representations without pre-training.
• Develop a dual-stage optimization framework that alternates radiance-field estimation and path planning.
• Eliminate depth supervision and pre-training requirements while using only color images.
• Incorporate collision avoidance, geometric information gain, and path efficiency into planning.

Proposed method

• Use DVGO as a hybrid implicit-explicit radiance field to enable fast coarse mapping and high-quality refinement.
• Formulate planning as a differentiable multi-objective optimization combining collision penalty, information gain, and path efficiency.
• Introduce termination-entropy based information gain computed from view frustums to guide NBV selection.
• Represent trajectories with quadratic Bézier curves to ensure smooth feasible motions.
• Operate online by incrementally updating the radiance field with new observations, without pre-training the NeRF model.

Experimental results

Research questions

• RQ1 Can an RGB-only, online NeRF-like framework achieve competitive active mapping and planning without pre-trained models or depth data?
• RQ2 How can termination probability entropy of 3D points observed from viewpoints be used as an information gain criterion for NBV selection?
• RQ3 What is the impact of a multi-objective planning loss on collision avoidance, information gain, and trajectory efficiency in active SLAM?
• RQ4 How does the proposed online framework compare to offline NeRF-based methods and existing active reconstruction approaches in rendering and reconstruction quality?

Key findings

• The full online ActiveRMAP method achieves PSNR 30.181, SSIM 0.942, LPIPS 0.063 on Synthetic-NeRF rendering, despite not using pre-training.
• Compared with offline NeRF-family baselines (NeRF: 31.01/0.947/0.081; Mip-NeRF: 33.09/0.961/0.043; DVGO: 31.82/0.955/0.055), the proposed method remains competitive in rendering quality while offering online operation and active view selection.
• ActiveRMAP outperforms several baselines in active reconstruction scenarios and provides competitive results against existing NeRF-based active methods (e.g., ActiveNeRF).
• The approach introduces an entropy-based information gain criterion that improves NBV selection over random, furthest-view, and uncertainty-based baselines in the evaluated settings.
• The planning module effectively balances collision avoidance, information gain, and path efficiency via a differentiable multi-objective loss, enabling online trajectory optimization.