[Paper Review] Deep Level Sets: Implicit Surface Representations for 3D Shape Inference

Mateusz Michalkiewicz, Jhony Kaesemodel Pontes+3 more|arXiv (Cornell University)|Jan 21, 2019

3D Shape Modeling and AnalysisEngineering30 references61 citations

TL;DR

The paper proposes an end-to-end framework that predicts 3D shapes as oriented level-set implicit surfaces, improving reconstruction accuracy over voxel representations and enabling flexible topology handling.

ABSTRACT

Existing 3D surface representation approaches are unable to accurately classify pixels and their orientation lying on the boundary of an object. Thus resulting in coarse representations which usually require post-processing steps to extract 3D surface meshes. To overcome this limitation, we propose an end-to-end trainable model that directly predicts implicit surface representations of arbitrary topology by optimising a novel geometric loss function. Specifically, we propose to represent the output as an oriented level set of a continuous embedding function, and incorporate this in a deep end-to-end learning framework by introducing a variational shape inference formulation. We investigate the benefits of our approach on the task of 3D surface prediction and demonstrate its ability to produce a more accurate reconstruction compared to voxel-based representations. We further show that our model is flexible and can be applied to a variety of shape inference problems.

Motivation & Objective

• Motivate improvement over boundary voxels by using implicit level-set representations for 3D shape inference.
• Develop an end-to-end trainable framework that predicts a level-set embedding whose zero level-set yields the object surface.
• Incorporate geometric regularizers (area, volume, orientation, smoothness) into a variational loss for robust surface inference.
• Demonstrate that level-set representations yield higher fidelity reconstructions than voxel occupancy at comparable network capacity.

Proposed method

• Represent the 3D output as the zero level set of an embedding function phi and optimise a variational loss over phi.
• Define energy functionals E_X and E_N to align the surface Gamma with data points and normals.
• Incorporate regularisers such as surface area E_area and volume E_vol, plus an E_sdf term to promote |grad(phi)|≈1.
• Use delta_epsilon and H_epsilon to convert continuous losses into grid-based approximations for differentiability.
• Train a simple 3D auto-encoder + CNN predictor framework with a shared 64D latent space, comparing level-set and voxel baselines.

Experimental results

Research questions

• RQ1 Can implicit level-set representations, when integrated with a variational loss, provide more accurate 3D shape inference than voxel-based methods?
• RQ2 How do geometric priors (area, volume) and normal/orientation terms affect reconstruction quality within a level-set framework?
• RQ3 Is the approach robust to topology changes and capable of high-fidelity surface detail from single-view inputs?

Key findings

• Implicit level-set representations yield higher IoU scores than voxel occupancy at 20^3 and 30^3 resolutions for most categories.
• Chamfer distance is generally lower for level-set representations, indicating closer surface reconstruction, with larger gains at 20^3 and reduced gaps at 30^3.
• The improvement of level-sets over voxels reduces as resolution increases, and some classes (e.g., chair) show smaller gains due to topology specifics.
• Qualitative results show finer geometric details and smoother surfaces when using the level-set approach compared to voxel baselines.
• The framework achieves comparable performance to voxel methods while offering improved boundary accuracy and topological flexibility.