[Paper Review] BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering

Yuanbo Xiangli, Linning Xu+6 more|arXiv (Cornell University)|Dec 10, 2021

Advanced Vision and Imaging8 citations

TL;DR

BungeeNeRF proposes a progressive neural radiance field that models extreme multi-scale scenes by training in stages, starting from distant views and incrementally adding detail as the model grows. It uses residual blocks with skip connections and multi-level supervision to maintain high-fidelity rendering across scales, significantly outperforming standard NeRF and Mip-NeRF on city, synthetic, and drone-captured scenes with large depth-of-field and resolution variations.

ABSTRACT

Neural radiance fields (NeRF) has achieved outstanding performance in modeling 3D objects and controlled scenes, usually under a single scale. In this work, we focus on multi-scale cases where large changes in imagery are observed at drastically different scales. This scenario vastly exists in real-world 3D environments, such as city scenes, with views ranging from satellite level that captures the overview of a city, to ground level imagery showing complex details of an architecture; and can also be commonly identified in landscape and delicate minecraft 3D models. The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results. To address these issues, we introduce BungeeNeRF, a progressive neural radiance field that achieves level-of-detail rendering across drastically varied scales. Starting from fitting distant views with a shallow base block, as training progresses, new blocks are appended to accommodate the emerging details in the increasingly closer views. The strategy progressively activates high-frequency channels in NeRF's positional encoding inputs and successively unfolds more complex details as the training proceeds. We demonstrate the superiority of BungeeNeRF in modeling diverse multi-scale scenes with drastically varying views on multiple data sources (city models, synthetic, and drone captured data) and its support for high-quality rendering in different levels of detail.

Motivation & Objective

• Address the challenge of modeling 3D scenes with extreme scale variations, such as satellite-to-ground-level views in cities.
• Overcome the limitations of standard NeRF and Mip-NeRF, which struggle with multi-scale scenes due to uniform treatment of input signals across scales.
• Enable consistent, high-fidelity rendering across all levels of detail—from coarse remote views to fine-grained close-ups—by learning a hierarchical scene representation.
• Design a training paradigm that grows both the model and training data progressively, ensuring each stage focuses on emerging details at increasing resolution.
• Support flexible level-of-detail rendering by allowing different output heads to render views at different scales using the same unified model.

Proposed method

• Progressive training: incrementally add closer views to the training set and append new residual blocks to the model at each stage, starting from the coarsest scale.
• Residual block architecture: each new block receives positional encoding inputs via a skip connection and predicts color and density residuals relative to the previous stage’s output.
• Multi-level supervision: each block’s output head is trained on the union of images from the coarsest scale up to its corresponding scale, ensuring full use of scale-specific data.
• Hierarchical feature learning: the skip connection enables reuse of high-frequency components in positional encoding, allowing deeper blocks to refine details in closer views.
• Progressive model growth: instead of deepening a single network, the model expands by appending new blocks, maintaining training stability and improving feature hierarchy.
• Output head specialization: each block produces a distinct level-of-detail rendering, enabling flexible, scale-aware inference without retraining.

Experimental results

Research questions

• RQ1 Can a neural radiance field effectively model scenes with extreme scale variations, such as those spanning satellite, aerial, and ground-level views?
• RQ2 How can a single neural rendering model maintain high-fidelity detail across vastly different levels of detail without compromising remote view completeness?
• RQ3 What architectural design enables progressive learning of multi-scale 3D representations while preserving features from earlier, coarser stages?
• RQ4 How does multi-level supervision and residual block design improve rendering quality compared to standard NeRF training on multi-scale data?
• RQ5 To what extent can a progressive training strategy reduce blur in close-ups and artifacts in distant views compared to baseline NeRF and Mip-NeRF?

Key findings

• BungeeNeRF achieves significantly better PSNR and visual quality than standard NeRF and Mip-NeRF on city scenes with extreme scale variations, especially in close-up details.
• The model maintains complete and artifact-free remote views, unlike Mip-NeRF, which often produces incomplete or blurry peripheral regions.
• Ablation studies show that both residual connections and skip connections are critical: removing either degrades performance, especially at closer scales.
• The residual connection enables later blocks to refine earlier outputs, improving geometry consistency and depth accuracy across scales.
• BungeeNeRF successfully renders complex multi-dive camera trajectories in city scenes, preserving fine details on multiple targets simultaneously.
• On UAV-captured real-world scenes with altitude changes from 24m to 76m, BungeeNeRF outperforms Mip-NeRF in novel view synthesis, demonstrating practical utility.