[论文解读] Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks

Junyi Pan, Xiaoguang Han+3 more|arXiv (Cornell University)|Sep 1, 2019

3D Shape Modeling and Analysis参考文献 36被引用 16

一句话总结

该论文提出了一种端到端的深度学习框架，用于从单视角图像进行3D网格重建，通过渐进式面片剪枝和形变动态修改网格拓扑，实现了从单个亏格为0的模板网格对复杂拓扑结构的高保真重建。该方法在重建精度和网格质量方面均优于当前最先进方法，尤其在拓扑结构复杂的形状上表现更优。

ABSTRACT

Reconstructing the 3D mesh of a general object from a single image is now possible thanks to the latest advances of deep learning technologies. However, due to the nontrivial difficulty of generating a feasible mesh structure, the state-of-the-art approaches often simplify the problem by learning the displacements of a template mesh that deforms it to the target surface. Though reconstructing a 3D shape with complex topology can be achieved by deforming multiple mesh patches, it remains difficult to stitch the results to ensure a high meshing quality. In this paper, we present an end-to-end single-view mesh reconstruction framework that is able to generate high-quality meshes with complex topologies from a single genus-0 template mesh. The key to our approach is a novel progressive shaping framework that alternates between mesh deformation and topology modification. While a deformation network predicts the per-vertex translations that reduce the gap between the reconstructed mesh and the ground truth, a novel topology modification network is employed to prune the error-prone faces, enabling the evolution of topology. By iterating over the two procedures, one can progressively modify the mesh topology while achieving higher reconstruction accuracy. Moreover, a boundary refinement network is designed to refine the boundary conditions to further improve the visual quality of the reconstructed mesh. Extensive experiments demonstrate that our approach outperforms the current state-of-the-art methods both qualitatively and quantitatively, especially for the shapes with complex topologies.

研究动机与目标

• 解决现有网格形变方法受限于固定模板拓扑结构、无法重建复杂拓扑的问题。
• 实现从单个亏格为0模板网格出发的端到端学习，支持动态拓扑适应的单视角3D网格重建。
• 通过统一框架整合拓扑修改与边界优化，提升网格质量与重建精度。
• 通过渐进式形变策略，克服面片剪枝中的边界平滑性与最优阈值选择等挑战。

提出的方法

• 级联形变网络预测顶点偏移量，以缩小重建网格与真实网格之间的差距。
• 新颖的拓扑修改网络估计面级重建误差，并剪除高误差面片，以实现网格拓扑的演化。
• 渐进式形变框架在形变与拓扑修改之间交替进行，逐步提升网格质量与拓扑保真度。
• 边界优化网络对剪枝产生的开放边界施加平滑性约束，以提升视觉质量。
• 框架采用可微的面片剪枝机制，使用可学习阈值τ，并通过最小化Chamfer距离进行优化。
• 该方法端到端训练，对不同初始网格类型（如球体或单位正方形）具有鲁棒性。

实验结果

研究问题

• RQ1 深度学习框架能否在从单张RGB图像进行重建时动态修改网格拓扑？
• RQ2 如何有效将面片剪枝集成到可微的网格重建流水线中，以提升拓扑精度？
• RQ3 在渐进式网格优化过程中，如何实现重建误差降低与拓扑演化的最佳平衡？
• RQ4 边界优化对剪枝后开放边界的视觉质量与几何一致性有何影响？

主要发现

• 所提方法在3D形状自编码任务中达到0.655的Chamfer距离，优于AtlasNet-25的0.765，证明了其更优的重建精度。
• 当τ = 0.1时，方法在正向与反向Chamfer距离之间实现良好平衡，显著降低多样化形状的重建误差。
• 消融实验表明，渐进式形变策略显著提升网格质量；若移除该策略，则导致几何失真与剪枝精度下降。
• 边界优化有效减少视觉伪影，生成更清晰、更逼真的网格，尤其在面片剪枝产生的开放边界处效果显著。
• 该方法在不同初始网格上泛化能力出色，在球体与单位正方形模板上均保持一致的高性能表现。
• 框架能够实现固定模板形变方法无法支持的复杂拓扑结构的高质量网格重建，如图1的定性对比所示。