[Paper Review] Overfit Neural Networks as a Compact Shape Representation.

Thomas Davies, Derek Nowrouzezahrai+1 more|arXiv (Cornell University)|Sep 17, 2020

3D Shape Modeling and AnalysisEngineering21 citations

TL;DR

This paper proposes Neural Implicits—overfit neural networks that represent 3D shapes as compact, high-accuracy signed distance fields (SDFs). By using a novel loss function, sampling strategy, and supervision protocol, the method achieves superior accuracy to triangle meshes at equal storage costs, demonstrating strong potential as a first-class implicit shape representation in computer graphics.

ABSTRACT

Neural networks have proven to be effective approximators of signed distance fields (SDFs) for solid 3D objects. While prior work has focused on the generalization power of such approximations, we instead explore their suitability as a compact - if purposefully overfit - SDF representation of individual shapes. Specifically, we ask whether neural networks can serve as first-class implicit shape representations in computer graphics. We call such overfit networks Neural Implicits. Similar to SDFs stored on a regular grid, Neural Implicits have fixed storage profiles and memory layout, but afford far greater accuracy. At equal storage cost, Neural Implicits consistently match or exceed the accuracy of irregularly-sampled triangle meshes. We achieve this with a combination of a novel loss function, sampling strategy and supervision protocol designed to facilitate robust shape overfitting. We demonstrate the flexibility of our representation on a variety of standard rendering and modelling tasks.

Motivation & Objective

• To investigate whether overfit neural networks can serve as compact, high-fidelity representations of individual 3D shapes.
• To address the limitation of existing neural SDF approximations that prioritize generalization over shape-specific accuracy.
• To develop a representation that combines the fixed memory layout of grid-based SDFs with the superior accuracy of neural networks.
• To enable robust shape overfitting through a tailored training protocol, loss function, and sampling strategy.
• To evaluate the flexibility and performance of the proposed representation in standard rendering and modeling tasks.

Proposed method

• Proposes a novel loss function specifically designed to encourage overfitting to a single target shape, minimizing deviation from the ground-truth SDF.
• Employs a specialized sampling strategy that focuses on regions of high geometric complexity or high error, improving learning efficiency and accuracy.
• Introduces a supervision protocol that enforces precise SDF supervision at sampled points, ensuring accurate distance field approximation.
• Trains fully connected neural networks to predict signed distances at query points, using backpropagation to minimize the proposed loss.
• Maintains a fixed storage profile and memory layout, similar to regular SDF grids, enabling efficient memory access and integration into graphics pipelines.
• Optimizes the network to achieve high accuracy on a single shape without generalization to other shapes, treating overfitting as a feature, not a bug.

Experimental results

Research questions

• RQ1 Can overfit neural networks serve as a compact and accurate implicit representation for individual 3D shapes?
• RQ2 How does the accuracy of overfit neural networks compare to irregularly sampled triangle meshes at equal storage costs?
• RQ3 What training components—loss function, sampling, supervision—are essential for achieving robust shape overfitting?
• RQ4 Can Neural Implicits be effectively used in standard rendering and modeling workflows?
• RQ5 Does the fixed memory layout of Neural Implicits enable efficient integration into existing graphics systems?

Key findings

• At equal storage cost, Neural Implicits consistently match or exceed the accuracy of irregularly-sampled triangle meshes in representing 3D shapes.
• The proposed loss function and supervision protocol enable robust overfitting, significantly reducing SDF prediction error on target shapes.
• The method maintains a fixed storage profile and memory layout, similar to grid-based SDFs, enabling efficient memory access and integration.
• Neural Implicits achieve higher accuracy than triangle meshes while preserving the compactness and regularity of implicit representations.
• The representation demonstrates flexibility and effectiveness across a range of standard rendering and modeling tasks.
• The results confirm that overfitting, when properly controlled, can be leveraged as a feature to create highly accurate, shape-specific implicit representations.