[Paper Review] A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsampling

Wentao Qu, Yuantian Shao+3 more|arXiv (Cornell University)|Dec 3, 2023

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes PUDM, a conditional denoising diffusion probabilistic model for point cloud upsampling that directly learns the gradient of dense point cloud data distribution using a dual mapping paradigm, eliminating the need for explicit upsampling modules. It achieves state-of-the-art performance in Chamfer Distance and Hausdorff Distance on PU1K and PUGAN benchmarks, with robustness to noise and capability for arbitrary-scale generation via learned rate factor parameterization.

ABSTRACT

Point cloud upsampling (PCU) enriches the representation of raw point clouds, significantly improving the performance in downstream tasks such as classification and reconstruction. Most of the existing point cloud upsampling methods focus on sparse point cloud feature extraction and upsampling module design. In a different way, we dive deeper into directly modelling the gradient of data distribution from dense point clouds. In this paper, we proposed a conditional denoising diffusion probability model (DDPM) for point cloud upsampling, called PUDM. Specifically, PUDM treats the sparse point cloud as a condition, and iteratively learns the transformation relationship between the dense point cloud and the noise. Simultaneously, PUDM aligns with a dual mapping paradigm to further improve the discernment of point features. In this context, PUDM enables learning complex geometry details in the ground truth through the dominant features, while avoiding an additional upsampling module design. Furthermore, to generate high-quality arbitrary-scale point clouds during inference, PUDM exploits the prior knowledge of the scale between sparse point clouds and dense point clouds during training by parameterizing a rate factor. Moreover, PUDM exhibits strong noise robustness in experimental results. In the quantitative and qualitative evaluations on PU1K and PUGAN, PUDM significantly outperformed existing methods in terms of Chamfer Distance (CD) and Hausdorff Distance (HD), achieving state of the art (SOTA) performance.

Motivation & Objective

• To address the limitations of existing point cloud upsampling methods that rely on indirect approximation via upsampling modules and CD loss supervision.
• To explore the use of probabilistic generative models—specifically conditional DDPM—for direct modeling of dense point cloud data distribution.
• To improve geometric detail recovery by leveraging dominant features and avoiding bias toward sparse input structures.
• To enable high-fidelity, arbitrary-scale point cloud generation during inference by incorporating scale prior knowledge through a learnable rate factor.
• To enhance robustness to noise by decoupling from CD loss and utilizing the inherent denoising architecture of DDPM.

Proposed method

• PUDM formulates point cloud upsampling as a conditional generation problem using a denoising diffusion probabilistic model (DDPM), where sparse point clouds serve as conditions.
• It models the reverse denoising process by learning the gradient of the data distribution from ground-truth dense point clouds, enabling direct fitting to complex geometry.
• A dual mapping paradigm is introduced, establishing mappings between: (1) generated sparse and input sparse point clouds, and (2) dense point clouds and noise, enhancing feature perception.
• The model parameterizes a rate factor during training to encode prior knowledge of the scale ratio between sparse and dense point clouds, enabling arbitrary-scale inference.
• The training objective is derived from the ELBO, minimizing the reconstruction loss between predicted and true noise in the diffusion process, expressed as $\mathbb{E}_{t,\epsilon}||\epsilon - \epsilon_\theta(\sqrt{1-\bar{\alpha}_t}\epsilon + \sqrt{\bar{\alpha}_t}x_0, t, C)||^2$.
• The auto-regressive nature of DDPM ensures intrinsic point-wise coordinate mapping, avoiding structural distortions common in module-based upsampling.

Experimental results

Research questions

• RQ1 Can a conditional denoising diffusion model effectively model the underlying data distribution of dense point clouds for upsampling without relying on explicit upsampling modules?
• RQ2 How does the dual mapping paradigm improve feature discrimination and geometric detail recovery in point cloud upsampling?
• RQ3 Can the incorporation of a learnable rate factor enable high-fidelity, arbitrary-scale point cloud generation during inference?
• RQ4 Does the DDPM-based approach exhibit superior robustness to noise compared to CD-loss-dependent methods?
• RQ5 To what extent does modeling the gradient of the data distribution lead to better reconstruction fidelity than indirect feature-based approximation?

Key findings

• PUDM achieves state-of-the-art performance on the PU1K and PUGAN benchmarks, significantly outperforming prior methods in both Chamfer Distance (CD) and Hausdorff Distance (HD).
• On PU1K, PUDM achieves a CD of 0.00088 and HD of 0.0021, demonstrating superior reconstruction accuracy.
• On PUGAN, PUDM achieves a CD of 0.00102 and HD of 0.0023, further confirming its SOTA performance.
• The model exhibits strong noise robustness, maintaining high-quality outputs even under noisy input conditions.
• Visual results show that PUDM successfully recovers fine geometric details—such as facial features in the red pig—where previous methods fail.
• The ablation study confirms that the dual mapping and rate factor parameterization are critical for performance, especially in preserving complex structures.