[Paper Review] Learning Implicit Fields for Generative Shape Modeling

Zhiqin Chen, Hao Zhang|arXiv (Cornell University)|Dec 6, 2018

3D Shape Modeling and Analysis51 references60 citations

TL;DR

Introduces IM-NET, an implicit-field decoder for 3D shapes that improves visual quality in autoencoding, generation, interpolation, and single-view reconstruction by predicting inside/outside status for points relative to a shape.

ABSTRACT

We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes. An implicit field assigns a value to each point in 3D space, so that a shape can be extracted as an iso-surface. IM-NET is trained to perform this assignment by means of a binary classifier. Specifically, it takes a point coordinate, along with a feature vector encoding a shape, and outputs a value which indicates whether the point is outside the shape or not. By replacing conventional decoders by our implicit decoder for representation learning (via IM-AE) and shape generation (via IM-GAN), we demonstrate superior results for tasks such as generative shape modeling, interpolation, and single-view 3D reconstruction, particularly in terms of visual quality. Code and supplementary material are available at https://github.com/czq142857/implicit-decoder.

Motivation & Objective

• Motivate implicit fields as a superior representation for generative 3D shapes compared to voxel-based decoders.
• Develop a simple implicit field decoder (IM-NET) that predicts inside/outside status for space points given a shape feature.
• Demonstrate IM-NET within autoencoders, GANs, and SVR pipelines to improve surface quality and topology handling.
• Show that implicit fields enable high-resolution sampling and smoother interpolation beyond training resolutions.

Proposed method

• Define an implicit field F(p) that assigns 0/1 to points p in space based on inside/outside status relative to a shape.
• Parameterize F by an MLP fθ that takes a 3D point p and a shape feature vector, trained as binary classifier with weighted MSE loss.
• Use progressive training on voxelized shapes (16^3 → 128^3) and sample points densely near surfaces with density-aware weights w_p.
• Embed IM-NET into IM-AE and IM-GAN by replacing traditional decoders with the implicit decoder to form IM-AE and IM-GAN.
• Apply IM-NET to 2D/3D shape generation, autoencoding, interpolation, and single-view 3D reconstruction, with meshes obtained via Marching Cubes.

Experimental results

Research questions

• RQ1 Can an implicit field decoder provide superior surface quality and topology handling compared to voxel-based decoders for 3D shape tasks?
• RQ2 How does incorporating point coordinates with shape features in IM-NET affect learning of inside/outside status and boundary quality?
• RQ3 Do IM-AE and IM-GAN improve autoencoding, generation, interpolation, and single-view reconstruction across multiple categories?
• RQ4 What are the trade-offs in training time and sampling resolution when using IM-NET versus CNN-based decoders?
• RQ5 Can IM-NET be extended effectively to 2D shapes and to SVR with high-quality surfaces?

Key findings

• IM-NET yields higher visual quality surfaces and better boundary definition than CNN-based decoders.
• Implicit fields allow sampling at arbitrary resolutions, producing sharper meshes beyond training resolution.
• IM-GAN and IM-AE outperform baselines in shape generation and interpolation with smoother topology changes.
• Across 3D and 2D domains, IM-GAN/IM-AE achieve competitive or superior visual metrics and qualitative results, especially in surface quality.
• Single-view reconstruction with IM-SVR demonstrates competitive performance against state-of-the-art SVR methods, with better surface fidelity.