[论文解读] Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers

Stephan R. Richter, Stefan Roth|arXiv (Cornell University)|Apr 29, 2018

Advanced Vision and Imaging参考文献 1被引用 68

一句话总结

本文提出 Matryoshka Networks 作为一种利用嵌套形状层来预测三维几何的方法。

ABSTRACT

In this paper, we develop novel, efficient 2D encodings for 3D geometry, which enable reconstructing full 3D shapes from a single image at high resolution. The key idea is to pose 3D shape reconstruction as a 2D prediction problem. To that end, we first develop a simple baseline network that predicts entire voxel tubes at each pixel of a reference view. By leveraging well-proven architectures for 2D pixel-prediction tasks, we attain state-of-the-art results, clearly outperforming purely voxel-based approaches. We scale this baseline to higher resolutions by proposing a memory-efficient shape encoding, which recursively decomposes a 3D shape into nested shape layers, similar to the pieces of a Matryoshka doll. This allows reconstructing highly detailed shapes with complex topology, as demonstrated in extensive experiments; we clearly outperform previous octree-based approaches despite having a much simpler architecture using standard network components. Our Matryoshka networks further enable reconstructing shapes from IDs or shape similarity, as well as shape sampling.

研究动机与目标

• 介绍一种称为 Matryoshka Networks 的新型网络架构，用于三维几何预测。
• 提出嵌套形状层作为高效三维形状表示的概念。
• 展示嵌套层如何实现对三维结构的更好推理。

提出的方法

• 提出一种使用嵌套形状层来表示三维几何的神经网络架构。
• 利用嵌套表示从输入数据推断出详细的三维结构。
• 概述用于优化嵌套层模型以进行几何预测的训练过程。

实验结果

研究问题

• RQ1 与传统表示相比，使用嵌套形状层表示三维几何的好处是什么？
• RQ2 在给定输入下，Matryoshka Networks 在预测三维几何方面的表现如何？
• RQ3 嵌套形状层对三维预测的效率和准确性有何影响？

主要发现

• 在提供的来源摘录中不可用。
• 提供的文本中不包含确切的定量结果和发现。