[Paper Review] A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

Zhaoyang Lyu, Zhifeng Kong+3 more|arXiv (Cornell University)|Dec 7, 2021

3D Shape Modeling and Analysis41 citations

TL;DR

Introduces a two-stage PDR framework that uses conditional DDPM (CGNet) to generate a coarse, uniform complete point cloud and an RFNet to refine it, achieving state-of-the-art completion while accelerating generation up to 50x with little quality loss.

ABSTRACT

3D point cloud is an important 3D representation for capturing real world 3D objects. However, real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to non-uniform point cloud generation. To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet). The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation. DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation. The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud. Furthermore, we develop a novel dual-path architecture for both networks. The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces and sharp details. Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion. Remarkably, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.

Motivation & Objective

• Motivate improved point cloud completion that yields uniform density and sharp geometric details.
• Leverage conditional denoising diffusion probabilistic models (DDPM) to generate coarse completions conditioned on partial observations.
• Design a dual-path CGNet and RFNet with novel modules to extract multi-level features and accurately manipulate 3D point locations.
• Demonstrate acceleration of diffusion-based generation without significant loss in quality via refinement.
• Provide extensive experiments on MVP, MVP-40, and Completion3D to establish state-of-the-art performance.

Proposed method

• Model point cloud completion as a conditional DDPM task with CGNet generating a coarse completion conditioned on incomplete data.
• Use a Rev refinement RFNet to refine the coarse output and improve surface quality.
• Employ a dual-path architecture with a Condition Feature Extraction subnet and a Denoise subnet, augmented by Feature Transfer modules.
• Introduce Point Adaptive Deconvolution (PA-Deconv) for upsampling and a PA-Deconv–based upsampling in the Refinement Network.
• In training, optimize a simplified MSE loss on predicted noise εθ from the DDPM formulation, avoiding Chamfer/EMD losses for the main DDPM path.
• During refinement, predict per-point displacements to densify and sharpen the final surface, supervised by Chamfer Distance to ground truth.

Experimental results

Research questions

• RQ1 Does a conditional DDPM-based framework improve the uniformity and quality of completed point clouds compared with prior methods?
• RQ2 Can a refinement network effectively enhance DDPM-generated coarse completions while preserving speed gains from diffusion acceleration?
• RQ3 How do novel modules (PA-Deconv, FT) and the dual-path architecture impact completion quality and efficiency?
• RQ4 What are the trade-offs between density uniformity (EMD) and local geometric detail (CD/visual precision) in this paradigm?

Key findings

• The PDR paradigm outperforms previous state-of-the-art methods on multiple benchmarks (MVP, MVP-40, Completion3D) across CD, EMD, and F1 metrics.
• DDPM-based conditioning yields more uniform point distributions (lower EMD) and high-quality surfaces compared to CD-focused training.
• RFNet enables up to 50x acceleration in the diffusion inference without substantial performance loss while maintaining high-quality completions.
• The dual-path CGNet/RFNet with PA-Deconv and Feature Transfer modules effectively leverages multi-level incomplete-point features to guide completion.
• A refinement network using per-point displacements further improves geometry and allows controllable upsampling of the final point cloud.