[论文解读] Learning to Infer Implicit Surfaces without 3D Supervision

Shichen Liu, Shunsuke Saito+2 more|arXiv (Cornell University)|Nov 2, 2019

3D Shape Modeling and Analysis被引用 89

一句话总结

翻译为中文：提出从单视图或多视图二维图像中，通过基于射线的场探测方法和几何正则化进行隐式三维表面无监督学习的方法，在无需3D监督的情况下实现具有高保真拓扑感知的形状。

ABSTRACT

Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. We demonstrate the effectiveness of our framework on the task of single-view image-based 3D shape digitization and show how we outperform state-of-the-art techniques both quantitatively and qualitatively.

研究动机与目标

• 在没有三维地面真相监督的情况下，激励从丰富的二维图像中学习高质量的三维形状。
• 利用隐式表面表示来处理任意拓扑结构和多分辨率细节。
• 通过基于射线的探测框架开发高效的图像对场监督。
• 引入几何正则化以强制局部表面平滑和合理的法向量。

提出的方法

• 通过从图像特征解码的神经隐式场定义占据概率。
• 使用带有球形区域的稀疏3D锚点来探测隐式场。
• 通过图像像素投射射线，并通过最大池化从相交的锚点聚合占据信息。
• 应用边界感知分配以减少靠近表面边界的标注歧义。
• 使用重要性采样将采样集中在图像轮廓和3D表面附近。
• 引入基于有限差分的几何正则化器，赋权以强调表面区域。

实验结果

研究问题

• RQ1 在没有3D地面真相数据的情况下，是否可以从二维监督学习到隐式表面？
• RQ2 如何高效地将二维图像线索与隐式三维场耦合，以推断出准确且拓扑结构复杂的形状？
• RQ3 哪些正则化策略最能在从二维轮廓训练的隐式表面上强制实现合理的几何？

主要发现

• 在 ShapeNet 上使用二维轮廓的无监督方法中实现了最先进的三维 IoU。
• 相比体素、点云和网格，产生了更高分辨率、拓扑灵活的重建。
• 证明所提出的带锚点与射线的场探测方法在准确性和细节方面有提升。
• 通过有限差分的几何正则化能够通过 p-范数实现可控的表面光滑度和法向量。
• 消融研究证实了边界感知分配和重要性采样对质量提升的好处。