[Paper Review] 3D Representation Methods: A Survey

Zhengren Wang|arXiv (Cornell University)|Oct 9, 2024

3D Surveying and Cultural Heritage5 citations

TL;DR

This paper surveys the development, strengths, and limitations of major 3D representation methods (voxel grids, point clouds, meshes, SDFs, NeRFs, 3D Gaussian splatting, Tri-plane, DMTet) and pivotal datasets, and outlines directions for future research.

ABSTRACT

The field of 3D representation has experienced significant advancements, driven by the increasing demand for high-fidelity 3D models in various applications such as computer graphics, virtual reality, and autonomous systems. This review examines the development and current state of 3D representation methods, highlighting their research trajectories, innovations, strength and weakness. Key techniques such as Voxel Grid, Point Cloud, Mesh, Signed Distance Function (SDF), Neural Radiance Field (NeRF), 3D Gaussian Splatting, Tri-Plane, and Deep Marching Tetrahedra (DMTet) are reviewed. The review also introduces essential datasets that have been pivotal in advancing the field, highlighting their characteristics and impact on research progress. Finally, we explore potential research directions that hold promise for further expanding the capabilities and applications of 3D representation methods.

Motivation & Objective

• Survey the evolution of 3D representation methods from geometric to neural and hybrid approaches.
• Summarize key techniques (voxel, point cloud, mesh, SDF, NeRF, 3D Gaussian splatting, Tri-plane, DMTet) and their trade-offs.
• Introduce influential datasets and analyze their impact on progress.
• Identify promising directions and open challenges for future work.

Proposed method

• Review and synthesize major 3D representation techniques including voxel grids, point clouds, meshes, SDFs, NeRFs, 3D Gaussian splatting, Tri-plane, and DMTet.
• Discuss hybrid approaches that combine multiple representations (e.g., DMTet, Tri-plane, 3D Gaussian Splatting).
• Highlight influential datasets and their roles in enabling advancements across tasks such as reconstruction, rendering, and scene understanding.
• Outline emerging research directions and practical considerations for future study.

Experimental results

Research questions

• RQ1 What are the main 3D representation techniques in use and their respective strengths and weaknesses?
• RQ2 How have datasets influenced progress across 3D representation methods?
• RQ3 What are the current trends and promising directions for future research in 3D representations?
• RQ4 How do hybrid methods combine different representations to address complex scenarios?

Key findings

• 3D representation evolved from explicit geometry (meshes, CSGL) to volume-based, point-based, implicit (SDF), and neural representations (NeRFs).
• Hybrid methods (e.g., DMTet, Tri-plane, 3D Gaussian Splatting) leverage strengths of multiple approaches for improved fidelity and efficiency.
• Neural radiance fields and their successors have driven significant progress in view synthesis and 3D reconstruction, with ongoing work to address efficiency, unbounded scenes, and real-world robustness.
• A broad ecosystem of influential datasets (e.g., ShapeNet, ModelNet, ScanNet, Pix3D, 3DPW, Objaverse) has underpinned progress across tasks like classification, reconstruction, and scene understanding.
• Emerging directions include real-time rendering (3D Gaussian splatting), scene-level generative models, and controllable/accurate representations for dynamic and articulated scenes.