[Paper Review] Improved Direct Voxel Grid Optimization for Radiance Fields Reconstruction

Cheng Sun, Min Sun+1 more|arXiv (Cornell University)|Jun 10, 2022

Medical Imaging Techniques and Applications17 citations

TL;DR

This paper proposes DVGOv2, an improved version of the Direct Voxel Grid Optimization framework for radiance field reconstruction, accelerating training by 2–3× through custom CUDA kernels and enabling forward-facing and unbounded inward-facing scene capture. It introduces an efficient O(N) distortion loss implementation that enhances both rendering quality and training speed, while maintaining high-fidelity results with a simple dense grid structure in PyTorch.

ABSTRACT

In this technical report, we improve the DVGO framework (called DVGOv2), which is based on Pytorch and uses the simplest dense grid representation. First, we re-implement part of the Pytorch operations with cuda, achieving 2-3x speedup. The cuda extension is automatically compiled just in time. Second, we extend DVGO to support Forward-facing and Unbounded Inward-facing capturing. Third, we improve the space time complexity of the distortion loss proposed by mip-NeRF 360 from O(N^2) to O(N). The distortion loss improves our quality and training speed. Our efficient implementation could allow more future works to benefit from the loss.

Motivation & Objective

• To accelerate training of radiance fields using explicit voxel grid representations without sacrificing reconstruction quality.
• To extend the original DVGO framework to support forward-facing and unbounded inward-facing scene reconstruction.
• To optimize the distortion loss computation from O(N²) to O(N) complexity for improved training efficiency and rendering quality.
• To maintain simplicity and ease of use via a PyTorch-based implementation while achieving high performance through targeted CUDA acceleration.
• To enable broader adoption of the distortion loss by providing an efficient, reusable implementation.

Proposed method

• Re-implementing key PyTorch operations in CUDA for 2–3× speedup, with just-in-time compilation for seamless integration.
• Extending the DVGO framework to support forward-facing and unbounded inward-facing scenes using a contracted space parameterization inspired by mip-NeRF 360.
• Introducing a novel O(N) computation scheme for the distortion loss by reworking the double summation in Eq. (1) into a linear pass over sampled points.
• Using a hybrid approach: explicit density and feature grids with a shallow MLP for view-dependent color, enabling fast inference and training.
• Applying a two-phase TV loss strategy: dense computation for first 10k iterations, then sparse computation only on active grid points to reduce overhead.
• Employing a cuboid voxel grid with a contracted space transformation (p=2 or p=∞) to better allocate grid points to background in unbounded scenes.

Experimental results

Research questions

• RQ1 Can the training speed of explicit radiance field methods be significantly improved without changing the underlying data structure?
• RQ2 Can the distortion loss be efficiently computed in O(N) time for point-based sampling, enabling its use in high-resolution, dense-grid methods?
• RQ3 Can the DVGO framework be extended to support forward-facing and unbounded inward-facing scenes while maintaining high reconstruction quality?
• RQ4 Does the O(N) distortion loss improve both training speed and rendering quality in dense grid-based radiance fields?
• RQ5 Can a simple, fully PyTorch-based framework with selective CUDA acceleration achieve performance comparable to state-of-the-art methods?

Key findings

• DVGOv2 achieves 2–3× speedup over the original DVGO through CUDA-optimized operations, with just-in-time compilation.
• The O(N) distortion loss implementation reduces computational complexity from O(N²) to O(N), improving training efficiency and rendering quality.
• On the LLFF dataset, DVGOv2 achieves PSNR of 26.34, SSIM of 0.838, and LPIPS of 0.197, outperforming baseline DVGO without distortion loss.
• On the mip-NeRF 360 dataset, DVGOv2 with p=∞ contracted space achieves PSNR of 25.42 and SSIM of 0.695, approaching NeRF++ performance with significantly reduced training time.
• The use of a cuboid contracted space (p=∞) improves results on the mip-NeRF 360 dataset, while p=2 performs better on Tanks&Temples, indicating sensitivity to dataset-specific photometric inconsistencies.
• Training time is reduced to 10.9 minutes on LLFF and 14.0 minutes on mip-NeRF 360 with a single 320³ grid, demonstrating efficiency despite limited resolution.