[Paper Review] NeuralRoom: Geometry-Constrained Neural Implicit Surfaces for Indoor Scene Reconstruction

Yusen Wang, Zongcheng Li+5 more|arXiv (Cornell University)|Oct 13, 2022

Advanced Vision and Imaging4 citations

TL;DR

NeuralRoom is a neural implicit surface reconstruction method that leverages multiview stereo depth and normal estimation priors to reduce shape-radiance ambiguity in indoor scene reconstruction. By integrating geometric constraints through perturbation-residual regularization, it achieves high-fidelity, complete reconstructions of both textured and texture-less regions, significantly improving state-of-the-art multiview methods on ScanNet.

ABSTRACT

We present a novel neural surface reconstruction method called NeuralRoom for reconstructing room-sized indoor scenes directly from a set of 2D images. Recently, implicit neural representations have become a promising way to reconstruct surfaces from multiview images due to their high-quality results and simplicity. However, implicit neural representations usually cannot reconstruct indoor scenes well because they suffer severe shape-radiance ambiguity. We assume that the indoor scene consists of texture-rich and flat texture-less regions. In texture-rich regions, the multiview stereo can obtain accurate results. In the flat area, normal estimation networks usually obtain a good normal estimation. Based on the above observations, we reduce the possible spatial variation range of implicit neural surfaces by reliable geometric priors to alleviate shape-radiance ambiguity. Specifically, we use multiview stereo results to limit the NeuralRoom optimization space and then use reliable geometric priors to guide NeuralRoom training. Then the NeuralRoom would produce a neural scene representation that can render an image consistent with the input training images. In addition, we propose a smoothing method called perturbation-residual restrictions to improve the accuracy and completeness of the flat region, which assumes that the sampling points in a local surface should have the same normal and similar distance to the observation center. Experiments on the ScanNet dataset show that our method can reconstruct the texture-less area of indoor scenes while maintaining the accuracy of detail. We also apply NeuralRoom to more advanced multiview reconstruction algorithms and significantly improve their reconstruction quality.

Motivation & Objective

• Address the challenge of shape-radiance ambiguity in neural implicit surface reconstruction for indoor scenes.
• Improve reconstruction quality in texture-less regions where multiview stereo fails.
• Integrate reliable geometric priors (depth and normal) to constrain neural optimization and avoid local minima.
• Enhance completeness and detail in flat, low-texture areas using perturbation-residual regularization.
• Enable performance gains for existing multiview reconstruction pipelines by integrating NeuralRoom as a post-processing refinement module.

Proposed method

• Use multiview stereo (MVS) results as depth priors to constrain the optimization space of the neural implicit surface.
• Integrate neural normal estimation outputs as geometric priors to guide surface geometry in texture-less regions.
• Apply perturbation-residual restrictions that enforce local surface consistency—sampling points in a neighborhood should have similar normals and distances to the camera center.
• Train a differentiable renderer with geometry-constrained optimization to ensure rendered views match input images.
• Use a two-branch MLP (Geo-MLP and Color-MLP) to predict signed distance and color from 3D coordinates, with geometric priors regularizing the optimization.
• Replace traditional depth estimation (e.g., COLMAP) with advanced learning-based methods (e.g., Atlas, NeuralRecon) to improve prior quality and further boost reconstruction performance.

Experimental results

Research questions

• RQ1 Can geometric priors from multiview stereo and normal estimation reduce shape-radiance ambiguity in neural implicit reconstruction of indoor scenes?
• RQ2 How can local surface consistency be enforced to improve completeness and smoothness in texture-less regions?
• RQ3 To what extent can NeuralRoom enhance the reconstruction quality of existing multiview reconstruction pipelines?
• RQ4 Does the integration of depth and normal priors lead to better generalization on complex indoor scenes with both textured and flat surfaces?
• RQ5 What is the impact of prior quality (e.g., COLMAP vs. learning-based methods) on the final reconstruction fidelity?

Key findings

• NeuralRoom achieves a 66.756 F-score on the ScanNet benchmark, significantly outperforming baseline methods like NeuS and Unisurf in reconstructing both textured and texture-less regions.
• The perturbation-residual regularization improves surface completeness and smoothness in flat regions, reducing artifacts common in standard implicit methods.
• When applied to advanced reconstruction pipelines (e.g., Atlas, NeuralRecon), NeuralRoom boosts F-score by up to 7.5% (from 68.347 to 73.003), demonstrating strong generalization and compatibility.
• Replacing COLMAP with learning-based depth estimation (e.g., Atlas) increases the F-score from 68.347 to 73.003, confirming the importance of high-quality priors.
• The method maintains high reconstruction quality even with reduced MLP capacity (4+2 layers), though more complex scenes require larger networks.
• NeuralRoom successfully reconstructs mirror-like and black surfaces that are typically missed by depth sensors, due to its RGB-only input and implicit learning.