[論文レビュー] Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction

Qiancheng Fu, Qingshan Xu+2 more|arXiv (Cornell University)|May 31, 2022

Advanced Vision and Imaging被引用数 81

ひとこと要約

Geo-Neus は多視ジオメトリ制約を用いて SDF のゼロレベルセットを明示的に最適化し、幾何学整合性のあるニューラル暗黙表面再構築を実現し、多視データセットで最先端手法を上回っている。

ABSTRACT

Recently, neural implicit surfaces learning by volume rendering has become popular for multi-view reconstruction. However, one key challenge remains: existing approaches lack explicit multi-view geometry constraints, hence usually fail to generate geometry consistent surface reconstruction. To address this challenge, we propose geometry-consistent neural implicit surfaces learning for multi-view reconstruction. We theoretically analyze that there exists a gap between the volume rendering integral and point-based signed distance function (SDF) modeling. To bridge this gap, we directly locate the zero-level set of SDF networks and explicitly perform multi-view geometry optimization by leveraging the sparse geometry from structure from motion (SFM) and photometric consistency in multi-view stereo. This makes our SDF optimization unbiased and allows the multi-view geometry constraints to focus on the true surface optimization. Extensive experiments show that our proposed method achieves high-quality surface reconstruction in both complex thin structures and large smooth regions, thus outperforming the state-of-the-arts by a large margin.

研究の動機と目的

• 多視点再構成における幾何学的整合性を持つニューラル暗黙表面の必要性を動機づける。
• ボリュームレンダリングと点ベースのSDFモデリング間のバイアスを特定する。
• sparse 3D点とMVSベースのフォトメトリック制約に導かれた明示的なSDF最適化を提案する。
• 薄い構造や滑らかな領域の表面再構築品質の改善を示す。

提案手法

• Theoretically analyze the gap between volume rendering integral and point-based SDF modeling.
• Directly locate the zero-level set of the SDF network to define the surface.
• Use sparse 3D points from Structure-from-Motion as explicit geometry supervision (view-aware SDF loss).
• Apply occlusion-aware sampling along view rays to identify the first surface intersection for supervision.
• Enforce geometry-consistent multi-view constraints via (a) surface intersection sampling and (b) photometric consistency using MVS plane-induced homographies and NCC-based loss.
• Train with a combined loss including color rendering, SDF supervision, and photometric consistency.

実験結果

リサーチクエスチョン

• RQ1 Does explicit SDF optimization via zero-level set supervision improve geometry accuracy over purely volume-rendering-based supervision?
• RQ2 Can sparse SFM points and photometric consistency constraints guide implicit surfaces toward geometry-consistent reconstructions across views?
• RQ3 How do thin structures and large smooth regions benefit from explicit SDF and MVS-based constraints?
• RQ4 What is the impact of removing volume-rendering biases on convergence speed and final geometry quality?

主な発見

Scan	IDR	NeuS	VolSDF	NeuS	NeuralWarp	colmap	Ours
24	1.63	1.15	1.14	1.37	0.49	0.45	0.375
37	1.87	0.95	1.26	1.21	0.71	0.91	0.537
40	0.63	0.80	0.81	0.73	0.38	0.37	0.336
55	0.48	0.39	0.49	0.40	0.38	0.37	0.357
63	1.04	1.26	1.25	1.20	0.79	0.90	0.800
65	0.79	0.72	0.70	0.70	0.81	1.00	0.454
69	0.77	0.69	0.72	0.72	0.82	0.54	0.408
83	1.33	0.94	1.29	1.01	1.20	1.22	1.032
97	1.16	1.14	1.18	1.16	1.06	1.08	0.843
105	0.76	0.77	0.70	0.82	0.68	0.64	0.548
106	0.67	0.66	0.66	0.66	0.66	0.48	0.460
110	0.90	1.35	1.08	1.69	0.74	0.59	0.473
114	0.42	0.39	0.42	0.39	0.41	0.32	0.294
118	0.51	0.51	0.61	0.49	0.63	0.45	0.355
122	0.53	0.52	0.55	0.51	0.51	0.43	0.345
mean	0.90	0.82	0.86	0.87	0.68	0.65	0.508

• Geo-Neus outperforms state-of-the-art neural implicit surface methods by a large margin on DTU scenes.
• It achieves higher surface quality for both complex thin structures and large smooth regions.
• Explicit SDF supervision with sparse 3D points improves geometry over baselines.
• Photometric consistency further boosts reconstruction of smooth regions and maintains fine details.
• Geo-Neus converges faster than NeuS, reducing training time from ~16 hours to ~10 hours.