[论文解读] F4Splat: Feed-Forward Predictive Densification for Feed-Forward 3D Gaussian Splatting

Injae Kim, Chaehyeon Kim+3 more|arXiv (Cornell University)|Mar 22, 2026

Computer Graphics and Visualization Techniques被引用 0

一句话总结

F4Splat 引入一个前馈 densification 预测器，能够自适应为 3D Gaussian Splatting 分配高斯原语，实现预算受控的非均匀高斯分布，并在无需对每个场景进行优化的情况下提升新视角合成效果。

ABSTRACT

Feed-forward 3D Gaussian Splatting methods enable single-pass reconstruction and real-time rendering. However, they typically adopt rigid pixel-to-Gaussian or voxel-to-Gaussian pipelines that uniformly allocate Gaussians, leading to redundant Gaussians across views. Moreover, they lack an effective mechanism to control the total number of Gaussians while maintaining reconstruction fidelity. To address these limitations, we present F4Splat, which performs Feed-Forward predictive densification for Feed-Forward 3D Gaussian Splatting, introducing a densification-score-guided allocation strategy that adaptively distributes Gaussians according to spatial complexity and multi-view overlap. Our model predicts per-region densification scores to estimate the required Gaussian density and allows explicit control over the final Gaussian budget without retraining. This spatially adaptive allocation reduces redundancy in simple regions and minimizes duplicate Gaussians across overlapping views, producing compact yet high-quality 3D representations. Extensive experiments demonstrate that our model achieves superior novel-view synthesis performance compared to prior uncalibrated feed-forward methods, while using significantly fewer Gaussians.

研究动机与目标

• Motivate reducing Gaussian redundancy in feed-forward 3D Gaussian Splatting by moving from uniform to adaptive allocation.
• Propose a densification-score guided framework to control the final Gaussian budget without retraining.
• Enable spatially adaptive Gaussian allocation that concentrates primitives on complex regions while avoiding duplicates across views.
• Train a model to predict densification signals from uncalibrated image sets, enabling single-pass reconstruction.
• Demonstrate state-of-the-art performance in uncalibrated feed-forward 3DGS with substantially fewer Gaussians.

提出的方法

• Provide a feed-forward network F_theta that outputs a set of Gaussians G and predicted camera parameters from context images and a target Gaussian budget N_G.
• Predict multi-scale Gaussian parameter maps and densification score maps to enable level-wise, threshold-guided allocation across scales.
• Use a geometry backbone (VGGT-like) with DINOv2 features to estimate camera parameters and features for Gaussian maps.
• Introduce a densification score supervised by backpropagated view-space gradients from rendering loss, enabling budget-guided allocation during training.
• Employ a budget-matching algorithm to select Gaussians across multi-scale maps so that the final count matches a user-specified budget.
• Train with novel views by aligning ground-truth target poses to the predicted coordinate system to promote generalization across viewpoints.

实验结果

研究问题

• RQ1 Can a feed-forward 3D Gaussian Splatting pipeline allocate Gaussians non-uniformly to match scene complexity under a fixed budget?
• RQ2 Does a densification-score guided allocation improve novel-view synthesis quality with fewer Gaussians compared to uniform allocation baselines?
• RQ3 Is it possible to train a model that predicts densification signals from uncalibrated image collections and control the final Gaussian budget without retraining?
• RQ4 How well does the method generalize to unseen datasets when trained on RE10K?
• RQ5 Can the approach operate effectively in pose-free and uncalibrated settings while maintaining rendering fidelity?

主要发现

Method	#GS (8 views)	LPIPS (8)	SSIM (8)	PSNR (8)	#GS (16 views)	LPIPS (16)	SSIM (16)	PSNR (16)	#GS (24 views)	LPIPS (24)	SSIM (24)	PSNR (24)
F4 Splat _tau+_	105K	0.142	0.847	25.26	210K	0.130	0.860	25.75	315K	0.128	0.862	25.85
F4 Splat _tau-_	447K	0.131	0.859	25.64	820K	0.120	0.869	26.10	1142K	0.119	0.870	26.18

• F4Splat achieves competitive or superior novel-view synthesis quality while using significantly fewer Gaussians than prior uncalibrated feed-forward methods.
• The method demonstrates non-uniform Gaussian allocation that concentrates primitives on fine detail regions and reduces redundancy in simple regions.
• Under the same Gaussian budget, F4Splat outperforms pose-free and uncalibrated baselines in LPIPS, SSIM, and PSNR across 8, 16, and 24 views.
• Budget control via the densification threshold tau allows maintaining fidelity while reducing the total Gaussian count, with explicit budget matching.
• Generalization tests show strong performance when transferring from RE10K to unseen ACID data, maintaining high-quality reconstructions with fewer Gaussians.