[Paper Review] AtlasNet: A Papier-M\\^ach\\'e Approach to Learning 3D Surface Generation

Thibault Groueix, Matthew Fisher+3 more|arXiv (Cornell University)|Feb 14, 2018

3D Shape Modeling and Analysis31 references177 citations

TL;DR

AtlasNet learns to generate 3D surfaces by composing multiple learnable 2D patches that map to a 3D surface, enabling high-resolution meshes and improved generalization over point-based baselines. It demonstrates auto-encoding and single-view reconstruction on ShapeNet with mesh outputs.

ABSTRACT

We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues. We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) auto-encoding shapes, and (ii) single-view reconstruction from a still image. We also provide results showing its potential for other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.

Motivation & Objective

• Motivate learning surface generation directly as a parametric 2D-to-3D mapping to produce continuous surfaces.
• Propose AtlasNet: a union of learnable 2D parameterizations that map to a 3D surface.
• Show that the approach yields high-fidelity meshes and scalable resolution without heavy memory use.
• Demonstrate auto-encoding and single-view reconstruction on ShapeNet and explore applications like morphing, parameterization, and correspondence.

Proposed method

• Represent a 3D shape as a collection of learnable parametric surface elements (patches).
• Decode a shape by concatenating a latent shape feature with 2D coordinates and passing through MLPs with ReLU activations to map unit-square samples to 3D points.
• Train using Chamfer distance between generated points and target surface samples, across N patches that cover the shape.
• Transfer a regular mesh from 2D patches to 3D to obtain meshes, enabling arbitrary-resolution sampling and texturing.
• Optionally enhance mesh quality via dense sampling followed by Poisson surface reconstruction (PSR).
• Use an encoder (PointNet for point clouds; ResNet-18 for images) to produce a latent representation, which conditions the patch-based decoders.

Experimental results

Research questions

• RQ1 Can a surface be effectively learned as a union of local 2D parameterizations that map to a 3D shape?
• RQ2 Does AtlasNet improve precision and generalization compared to point-based generation baselines on ShapeNet?
• RQ3 Can the patch-based surface generation produce high-resolution meshes suitable for single-view reconstruction and texturing?
• RQ4 How does the number of patches affect fidelity, topology handling, and generalization to unseen categories?

Key findings

• AtlasNet outperforms a points-based baseline on Chamfer distance and METRO mesh distance in auto-encoding on ShapeNet.
• Increasing the number of patches improves reconstruction quality and generalization to new categories.
• The model can produce high-resolution meshes (e.g., tens of thousands of points) directly without prohibitive memory costs.
• Single-view reconstruction from RGB images yields meshes that preserve fine surface details better than some voxel/octree-based baselines.
• The approach supports applications like shape interpolation, correspondences, and texture parameterization via the inferred UV maps.