[논문 리뷰] A Survey on 3D Gaussian Splatting

Guikun Chen, Wenguan Wang|arXiv (Cornell University)|2024. 01. 08.

Image Enhancement Techniques인용 수 38

한 줄 요약

이 설문은 실시간 렌더링 가능하고 편집 가능한 장면 표현을 가능하게 하는 명시적 radiance-field 표현인 3D Gaussian Splatting(3D GS)에 대한 체계적 개요를 제공하고, 주요 모델들을 작업별로 비교하며 도전과제와 향후 방향을 제시합니다.

ABSTRACT

3D Gaussian splatting (GS) has emerged as a transformative technique in radiance fields. Unlike mainstream implicit neural models, 3D GS uses millions of learnable 3D Gaussians for an explicit scene representation. Paired with a differentiable rendering algorithm, this approach achieves real-time rendering and unprecedented editability, making it a potential game-changer for 3D reconstruction and representation. In the present paper, we provide the first systematic overview of the recent developments and critical contributions in 3D GS. We begin with a detailed exploration of the underlying principles and the driving forces behind the emergence of 3D GS, laying the groundwork for understanding its significance. A focal point of our discussion is the practical applicability of 3D GS. By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond. This is complemented by a comparative analysis of leading 3D GS models, evaluated across various benchmark tasks to highlight their performance and practical utility. The survey concludes by identifying current challenges and suggesting potential avenues for future research. Through this survey, we aim to provide a valuable resource for both newcomers and seasoned researchers, fostering further exploration and advancement in explicit radiance field.

연구 동기 및 목표

• Explain the motivations and fundamentals behind 3D Gaussian Splatting (3D GS).
• Summarize the optimization, rendering, and density-control techniques powering 3D GS.
• Survey applications and task performance of 3D GS across domains like SLAM, dynamic scenes, and AIGC.
• Highlight current challenges and potential future research directions in 3D GS.

제안 방법

• Describe the forward rendering process of 3D GS via splatting of 3D Gaussians onto the image plane.
• Explain how frustum culling, tiling, and Gaussian replication enable parallel, real-time rendering.
• Present the differentiable rendering framework where Gaussian properties (center, covariance, color, opacity) are learned.
• Outline the parameter optimization using a quaternion-based representation to ensure valid covariance matrices.
• Discuss density control through point densification and pruning to balance accuracy and efficiency.

실험 결과

연구 질문

• RQ1 How can 3D Gaussians efficiently represent complex scenes for real-time rendering?
• RQ2 What are effective optimization strategies to learn and adjust Gaussians and their densities?
• RQ3 How does 3D GS compare to implicit NeRF-based methods on standard tasks and benchmarks?
• RQ4 What are the practical limitations and open challenges of 3D GS in various applications (SLAM, dynamic scenes, AIGC)?

주요 결과

• 3D GS enables real-time rendering with an explicit, learnable scene representation based on millions of 3D Gaussians.
• The rendering pipeline uses frustum culling, splatting, and tile-based parallelization to achieve efficiency.
• Gaussian parameters are learned via back-propagation, with covariance reconstructed from rotation and scale to ensure valid statistical properties.
• Density control through densification and pruning maintains representation quality while managing resource use.
• The survey covers applications across SLAM, dynamic scenes, autonomous driving, and AI-generated content, and analyzes performance and open challenges.