[Paper Review] CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization

Junsheng Zhou, Baorui Ma+4 more|arXiv (Cornell University)|Oct 6, 2022

3D Shape Modeling and Analysis28 citations

TL;DR

CAP-UDF learns a continuous unsigned distance function directly from raw point clouds by moving queries toward the surface under a consistency-aware field loss, enabling direct surface extraction from the gradient field without ground-truth distances. It progressively refines surfaces and demonstrates state-of-the-art results on synthetic and real scans.

ABSTRACT

Surface reconstruction for point clouds is an important task in 3D computer vision. Most of the latest methods resolve this problem by learning signed distance functions from point clouds, which are limited to reconstructing closed surfaces. Some other methods tried to represent open surfaces using unsigned distance functions (UDF) which are learned from ground truth distances. However, the learned UDF is hard to provide smooth distance fields due to the discontinuous character of point clouds. In this paper, we propose CAP-UDF, a novel method to learn consistency-aware UDF from raw point clouds. We achieve this by learning to move queries onto the surface with a field consistency constraint, where we also enable to progressively estimate a more accurate surface. Specifically, we train a neural network to gradually infer the relationship between queries and the approximated surface by searching for the moving target of queries in a dynamic way. Meanwhile, we introduce a polygonization algorithm to extract surfaces using the gradients of the learned UDF. We conduct comprehensive experiments in surface reconstruction for point clouds, real scans or depth maps, and further explore our performance in unsupervised point normal estimation, which demonstrate non-trivial improvements of CAP-UDF over the state-of-the-art methods.

Motivation & Objective

• Motivate surface reconstruction from raw point clouds without ground-truth distances or large training data.
• Introduce a consistency-aware unsigned distance function (UDF) that guides queries toward the surface.
• Propose a progressive surface approximation strategy to refine the surface over stages.
• Develop a surface extraction method directly from the gradient field of the learned UDF.
• Demonstrate state-of-the-art reconstruction accuracy on synthetic shapes, real scans, and scenes.

Proposed method

• Predict unsigned distance f(q) for 3D queries q via an MLP network.
• Move each query q by a step along the negative normalized gradient direction using stride f(q).
• Train with a consistency-aware loss that compares moved queries z with the closest surface point in the evolving point set via Chamfer distance.
• Progressively update the reference point set with moved queries to refine local geometry in subsequent stages.
• Extract surfaces directly from the gradient field by adapted gradient-based marching-cubes-like procedure.
• Optionally refine the initial mesh by adjusting vertex positions along the learned distance field using local UDF values.

Experimental results

Research questions

• RQ1 Can a UDF learned directly from raw point clouds produce smooth distance fields near surfaces without ground-truth distances?
• RQ2 Does a consistency-aware loss stabilize training and prevent field distortion when learning UDFs from discrete point clouds?
• RQ3 Can surfaces with open/topology-rich structures be extracted directly from the gradient field of a learned UDF without BPA-based post-processing?
• RQ4 Does progressive surface approximation improve detail capture and convergence compared to single-stage training?
• RQ5 How does CAP-UDF perform on synthetic and real scanned data relative to state-of-the-art unsigned distance approaches?

Key findings

• CAP-UDF achieves continuous unsigned distance fields from raw point clouds without ground-truth distances or large-scale training data.
• A consistency-aware field loss mitigates conflicting optimization directions and yields more accurate and smooth near-surface fields.
• Progressive stages with moved-query priors improve local detail and surface fidelity over iteration.
• Direct gradient-based surface extraction from the learned UDF yields high-quality surfaces with arbitrary topology, avoiding BPA.
• Empirical results show state-of-the-art performance on ShapeNet cars (synthetic), MGD real scans, SRB real scans, and OnSurf scene benchmarks across multiple metrics.
• Ablation studies confirm the effectiveness of the consistency loss, progressive stages, and surface extraction method.