[Paper Review] Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces

Baorui Ma, Zhizhong Han+2 more|arXiv (Cornell University)|Nov 26, 2020

3D Shape Modeling and AnalysisEngineering63 references47 citations

TL;DR

Neural-Pull trains a neural network to predict signed distance values and gradients from point clouds and uses a differentiable pulling operation to move query points onto the surface, yielding high-quality SDFs for surface reconstruction and single-image reconstruction.

ABSTRACT

Reconstructing continuous surfaces from 3D point clouds is a fundamental operation in 3D geometry processing. Several recent state-of-the-art methods address this problem using neural networks to learn signed distance functions (SDFs). In this paper, we introduce extit{Neural-Pull}, a new approach that is simple and leads to high quality SDFs. Specifically, we train a neural network to pull query 3D locations to their closest points on the surface using the predicted signed distance values and the gradient at the query locations, both of which are computed by the network itself. The pulling operation moves each query location with a stride given by the distance predicted by the network. Based on the sign of the distance, this may move the query location along or against the direction of the gradient of the SDF. This is a differentiable operation that allows us to update the signed distance value and the gradient simultaneously during training. Our outperforming results under widely used benchmarks demonstrate that we can learn SDFs more accurately and flexibly for surface reconstruction and single image reconstruction than the state-of-the-art methods.

Motivation & Objective

• Address the challenge of learning signed distance functions directly from raw 3D point clouds without ground truth signed distances.
• Overcome limitations of unsigned-distance-based approaches by enabling direct SDF learning from point clouds.
• Propose a differentiable space-pulling mechanism that uses predicted distances and gradients to align query points with the surface.
• Demonstrate improved surface reconstruction accuracy on standard benchmarks and extend to single-image reconstruction.

Proposed method

• Train a neural network to predict the signed distance value and its gradient at a 3D query location, given a point cloud as condition.
• Define a differentiable pulling operation that moves each query location toward its nearest surface point using the predicted distance and gradient, with a step proportional to the distance.
• Pull q_i to t_i' = q_i - f(c, q_i) * grad f(c, q_i) / ||grad f(c, q_i)||, and handle sign to move along or against the gradient.
• Sample query locations around surface points via an isotropic Gaussian distribution, with adaptive variance tied to local point density.
• Optimize using a squared loss between the pulled query location and the nearest surface point, effectively encouraging convergence to a signed distance field.
• Initialize network parameters with geometric network initialization to approximate a sphere's SDF and train with Adam for 2500 epochs.

Experimental results

Research questions

• RQ1 Can a neural network learn a true signed distance function directly from raw point clouds without ground-truth SDF supervision?
• RQ2 Does the differentiable space-pulling operation, leveraging predicted distances and gradients, lead to more accurate surface reconstruction than unsigned-distance or gradient-regularized methods?
• RQ3 How does sampling query locations near the surface affect learning efficiency and final SDF accuracy?
• RQ4 How robust is Neural-Pull to noisy point clouds or varying point density when learning SDFs?
• RQ5 Can the method extend to single-image reconstruction by treating images as conditioning inputs?

Key findings

• Neural-Pull achieves superior surface reconstruction accuracy on ABC, FAMOUS, and ShapeNet-derived benchmarks compared to several state-of-the-art methods (mean L2-CD improvements shown in the reported tables).
• The method learns SDFs directly from point clouds without ground-truth signed distances and outperforms unsigned-distance-based approaches.
• A differentiable pulling operation that uses both the predicted SDF value and its gradient leads to effective optimization and convergence toward valid SDFs.
• Query locations sampled near the surface and the geometric network initialization (GNI) significantly improve learning; space-only sampling or removing GNI degrades performance.
• Neural-Pull demonstrates strong generalization to single-image reconstruction and to real-image inputs, yielding high-fidelity surface reconstructions.