[Paper Review] Evaluating Neural Radiance Fields (NeRFs) for 3D Plant Geometry Reconstruction in Field Conditions

Muhammad Arshad, Talukder Z. Jubery+6 more|arXiv (Cornell University)|Feb 15, 2024

Surface Roughness and Optical Measurements4 citations

TL;DR

This paper evaluates Neural Radiance Fields (NeRFs) for 3D reconstruction of plants in field conditions, demonstrating that NeRFs achieve a 74.65% F1 score with 30 minutes of GPU training. The study compares NeRF variants under increasing complexity, using LiDAR scans as ground truth, and identifies Nerfacto as the most effective due to its advanced sampling strategy and pose refinement.

ABSTRACT

We evaluate different Neural Radiance Fields (NeRFs) techniques for the 3D reconstruction of plants in varied environments, from indoor settings to outdoor fields. Traditional methods usually fail to capture the complex geometric details of plants, which is crucial for phenotyping and breeding studies. We evaluate the reconstruction fidelity of NeRFs in three scenarios with increasing complexity and compare the results with the point cloud obtained using LiDAR as ground truth. In the most realistic field scenario, the NeRF models achieve a 74.6% F1 score after 30 minutes of training on the GPU, highlighting the efficacy of NeRFs for 3D reconstruction in challenging environments. Additionally, we propose an early stopping technique for NeRF training that almost halves the training time while achieving only a reduction of 7.4% in the average F1 score. This optimization process significantly enhances the speed and efficiency of 3D reconstruction using NeRFs. Our findings demonstrate the potential of NeRFs in detailed and realistic 3D plant reconstruction and suggest practical approaches for enhancing the speed and efficiency of NeRFs in the 3D reconstruction process.

Motivation & Objective

• To assess the feasibility and performance of NeRF-based 3D reconstruction for complex plant geometries in real-world field environments.
• To compare multiple NeRF variants under increasing environmental and structural complexity, from indoor to outdoor field settings.
• To establish a benchmark using LiDAR-derived point clouds as ground truth for evaluating NeRF accuracy in plant morphology.
• To develop a reusable evaluation framework and dataset for future research in agricultural NeRF applications.

Proposed method

• The study employs three evaluation scenarios: controlled indoor, semi-controlled outdoor, and realistic field conditions, progressively increasing complexity.
• NeRF models are trained using multi-view RGB images captured from diverse camera poses, with training durations optimized for field deployment.
• A LiDAR-derived point cloud is used as the ground truth for quantitative evaluation, enabling F1 score and structural similarity comparisons.
• The evaluation framework includes metrics such as F1 score, structural similarity, and inference speed, with ablation studies on sampling strategies.
• Nerfacto’s proposal network and density field are leveraged to improve sampling efficiency and rendering quality in complex plant scenes.
• The method incorporates camera pose refinement and piecewise sampling to enhance detail capture in both near and far regions of the scene.

Experimental results

Research questions

• RQ1 Can NeRFs achieve accurate and detailed 3D reconstruction of plant geometry in uncontrolled field conditions?
• RQ2 How do different NeRF variants (e.g., Instant-NGP, Nerfacto) compare in terms of reconstruction quality and inference efficiency under field constraints?
• RQ3 What is the impact of sampling strategy and pose refinement on the fidelity of plant structure reconstruction?
• RQ4 To what extent can NeRFs outperform traditional photogrammetric methods in capturing fine plant details like venation and branching patterns?

Key findings

• In the most realistic field scenario, NeRF models achieved a 74.65% F1 score after only 30 minutes of training on a GPU, demonstrating strong performance under real-world conditions.
• Nerfacto outperformed other NeRF variants due to its advanced sampling strategy, including proposal networks and camera pose refinement, which enhanced image crispness and detail accuracy.
• The use of a LiDAR-derived point cloud as ground truth enabled reliable quantitative evaluation, confirming NeRFs’ ability to reconstruct fine plant structures with high fidelity.
• Instant-NGP achieved faster inference but produced blurrier results due to less precise sampling, particularly in regions with high geometric complexity.
• The study provides a publicly available dataset of RGB images and corresponding LiDAR scans, serving as a benchmark for future NeRF research in agriculture.
• The developed evaluation framework enables systematic comparison of NeRF models, supporting reproducible and scalable research in plant 3D reconstruction.