[Paper Review] PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths

Xin Wen, Xiang Peng+5 more|arXiv (Cornell University)|Dec 7, 2020

3D Shape Modeling and Analysis49 references17 citations

TL;DR

PMP-Net proposes a novel point cloud completion method that formulates shape completion as a multi-step point deformation process, learning unique point moving paths (PMP) under Earth Mover's Distance (EMD) regularization to improve structural fidelity. By using a coarse-to-fine path search with a Recurrent Path Aggregation (RPA) module, it achieves state-of-the-art performance on Completion3D and PCN datasets, significantly outperforming generative baselines.

ABSTRACT

The task of point cloud completion aims to predict the missing part for an incomplete 3D shape. A widely used strategy is to generate a complete point cloud from the incomplete one. However, the unordered nature of point clouds will degrade the generation of high-quality 3D shapes, as the detailed topology and structure of discrete points are hard to be captured by the generative process only using a latent code. In this paper, we address the above problem by reconsidering the completion task from a new perspective, where we formulate the prediction as a point cloud deformation process. Specifically, we design a novel neural network, named PMP-Net, to mimic the behavior of an earth mover. It moves each point of the incomplete input to complete the point cloud, where the total distance of point moving paths (PMP) should be shortest. Therefore, PMP-Net predicts a unique point moving path for each point according to the constraint of total point moving distances. As a result, the network learns a strict and unique correspondence on point-level, which can capture the detailed topology and structure relationships between the incomplete shape and the complete target, and thus improves the quality of the predicted complete shape. We conduct comprehensive experiments on Completion3D and PCN datasets, which demonstrate our advantages over the state-of-the-art point cloud completion methods.

Motivation & Objective

• To address the limitations of generative models in capturing detailed topology and structure in incomplete point clouds.
• To establish strict, unique point-level correspondences between incomplete and complete shapes for improved completion quality.
• To reformulate point cloud completion as a differentiable point deformation process rather than direct generation.
• To enhance structural fidelity by learning optimal, shortest total path movements using EMD-inspired regularization.
• To enable progressive refinement of point trajectories through a coarse-to-fine, multi-step path search strategy.

Proposed method

• PMP-Net formulates point cloud completion as a multi-step deformation process, where each point in the incomplete cloud is moved toward its corresponding position in the complete shape.
• It introduces a novel Recurrent Path Aggregation (RPA) module inspired by GRU to memorize and aggregate historical movement paths for each point.
• The network uses a coarse-to-fine path search strategy with decreasing search radii across steps to refine movement predictions progressively.
• It regularizes the learning process using a Point-Moving-Path (PMP) loss based on Earth Mover’s Distance (EMD), ensuring the total path length is minimized and promoting unique, stable correspondences.
• The model predicts both direction and length of movement at each step, conditioned on current position and aggregated historical path information.
• The architecture is trained end-to-end to minimize the PMP loss, which enforces geometric consistency and structural accuracy in the final completed point cloud.

Experimental results

Research questions

• RQ1 Can learning explicit point moving paths improve the structural fidelity of point cloud completion compared to direct generation?
• RQ2 How does EMD-based regularization on total path length affect the uniqueness and quality of predicted point correspondences?
• RQ3 What is the optimal number of steps and search radius scaling strategy for multi-step path refinement in point cloud deformation?
• RQ4 How does incorporating recurrent history aggregation improve the accuracy of path prediction over single-step or non-recurrent approaches?
• RQ5 To what extent does the coarse-to-fine path search strategy enhance completion quality compared to a single-step deformation?

Key findings

• PMP-Net achieves state-of-the-art performance on the Completion3D benchmark, with a mean Chamfer Distance (CD) of 11.58, outperforming the baseline.
• The ablation study shows that removing the PMP loss increases the mean CD to 13.66, confirming its critical role in improving structural accuracy.
• Using a 3-step path search with decreasing radius [1.0, 0.1, 0.01] yields the best performance, while 4 steps show diminishing returns due to redundancy.
• The coarse-to-fine strategy with decreasing radius [1.0, 0.1, 0.01] outperforms both fixed-radius [1.0,1.0,1.0] and slower-decaying [1.0,0.5,0.25] strategies.
• Visual analysis confirms that early-stage predictions with coarse-to-fine search are more complete and structured, enabling better refinement in later steps.
• The RPA module significantly improves performance over non-recurrent baselines, with a 1.7% reduction in mean CD compared to a simple RNN-based path aggregator.