[Paper Review] 3D Density-Gradient based Edge Detection on Neural Radiance Fields (NeRFs) for Geometric Reconstruction

Miriam Jäger, Boris Jutzi|arXiv (Cornell University)|Sep 26, 2023

Optical measurement and interference techniquesComputer Science3 citations

TL;DR

This paper proposes a 3D density-gradient-based edge detection method for geometric reconstruction from Neural Radiance Fields (NeRFs), leveraging Sobel, Canny, and Laplacian of Gaussian filters on voxelized density fields to extract edges independent of absolute density values. The Canny filter achieves the best balance of completeness and correctness, producing nearly gapless, high-accuracy 3D reconstructions without relying on empirical global density thresholds.

ABSTRACT

Generating geometric 3D reconstructions from Neural Radiance Fields (NeRFs) is of great interest. However, accurate and complete reconstructions based on the density values are challenging. The network output depends on input data, NeRF network configuration and hyperparameter. As a result, the direct usage of density values, e.g. via filtering with global density thresholds, usually requires empirical investigations. Under the assumption that the density increases from non-object to object area, the utilization of density gradients from relative values is evident. As the density represents a position-dependent parameter it can be handled anisotropically, therefore processing of the voxelized 3D density field is justified. In this regard, we address geometric 3D reconstructions based on density gradients, whereas the gradients result from 3D edge detection filters of the first and second derivatives, namely Sobel, Canny and Laplacian of Gaussian. The gradients rely on relative neighboring density values in all directions, thus are independent from absolute magnitudes. Consequently, gradient filters are able to extract edges along a wide density range, almost independent from assumptions and empirical investigations. Our approach demonstrates the capability to achieve geometric 3D reconstructions with high geometric accuracy on object surfaces and remarkable object completeness. Notably, Canny filter effectively eliminates gaps, delivers a uniform point density, and strikes a favorable balance between correctness and completeness across the scenes.

Motivation & Objective

• To address the limitations of global density thresholding in NeRF-based 3D reconstruction, which often leads to noisy, incomplete, or scene-dependent results.
• To explore the use of density gradients—derived from first and second derivatives—as a robust alternative to absolute density values for edge detection.
• To improve geometric accuracy and completeness in 3D reconstructions by leveraging anisotropic, neighborhood-based gradient filtering on voxelized density fields.
• To evaluate the performance of different 3D edge detection filters (Sobel, Canny, LoG) in the context of NeRF-based reconstruction.
• To demonstrate that gradient-based methods reduce dependency on empirical threshold tuning while maintaining or improving reconstruction quality.

Proposed method

• The method voxelizes the NeRF’s density field into a 3D grid to enable spatial gradient computation.
• It applies 3D edge detection filters—Sobel, Canny, and Laplacian of Gaussian (LoG)—to compute first and second derivatives of density values across neighboring voxels.
• The gradients are based on relative density differences, making them independent of absolute density magnitudes and thus robust across varying NeRF configurations and scenes.
• The resulting edge maps are used to extract geometric surfaces, with the Canny filter specifically designed to minimize gaps and ensure uniform point density.
• The approach processes the entire voxelized density field, enabling anisotropic handling of density variations without ray-specific sampling.
• Post-processing may involve surface extraction (e.g., convex hull) to remove internal artifacts caused by processing the full volume.

Figure 1: Density Gradient e.g. for 1D on a ray. The illustrations display the characteristics of the density values exemplarily as they would occur during ray tracing into the direction of an object. From top to bottom: An ideal edge in a binary non-object to object space, the raw density values, t

Experimental results

Research questions

• RQ1 Can density gradients from first and second derivatives improve geometric completeness and accuracy in NeRF-based 3D reconstruction compared to global density thresholding?
• RQ2 How do different 3D edge detection filters (Sobel, Canny, LoG) perform in terms of completeness, correctness, and gap formation across diverse scenes?
• RQ3 To what extent are gradient-based methods independent of absolute density values and thus robust across varying NeRF hyperparameters and network configurations?
• RQ4 Does the Canny filter provide a superior balance between correctness and completeness compared to other filters and thresholding strategies?
• RQ5 Can gradient-based reconstruction eliminate the need for empirical threshold tuning while maintaining or improving reconstruction quality?

Key findings

• The Canny filter achieved the highest geometric completeness and correctness, producing nearly gapless reconstructions across all scenes in the DTU benchmark.
• Quantitative results showed the Canny filter achieved a mean reconstruction error of 3.89 mm (reference-to-data), outperforming Sobel (3.97 mm) and LoG (3.52 mm) in overall accuracy.
• The Canny filter maintained a balanced trade-off between correctness and completeness, with a mean error of 2.85 mm on the reference-to-data metric, outperforming global thresholds that often sacrifice correctness for completeness.
• Sobel and LoG filters also showed strong performance, but the Canny filter consistently reduced gaps and ensured uniform point density across object surfaces.
• The method demonstrated independence from absolute density values, as gradients rely only on relative neighboring differences, reducing dependency on NeRF-specific hyperparameters.
• Despite high surface accuracy, internal artifacts (points within objects) were observed due to full-volume processing, suggesting a need for surface-only extraction methods like convex hulls.