[Paper Review] Non-Rigid Point Set Registration Networks

Wang, Lingjing, Chen Jian-chun+2 more|arXiv (Cornell University)|Apr 2, 2019

3D Shape Modeling and Analysis26 references33 citations

TL;DR

PR-Net learns a registration pattern from training data to directly predict non-rigid transformations for unseen point sets, achieving fast, robust registration without iterative optimization.

ABSTRACT

Point set registration is defined as a process to determine the spatial transformation from the source point set to the target one. Existing methods often iteratively search for the optimal geometric transformation to register a given pair of point sets, driven by minimizing a predefined alignment loss function. In contrast, the proposed point registration neural network (PR-Net) actively learns the registration pattern as a parametric function from a training dataset, consequently predict the desired geometric transformation to align a pair of point sets. PR-Net can transfer the learned knowledge (i.e. registration pattern) from registering training pairs to testing ones without additional iterative optimization. Specifically, in this paper, we develop novel techniques to learn shape descriptors from point sets that help formulate a clear correlation between source and target point sets. With the defined correlation, PR-Net tends to predict the transformation so that the source and target point sets can be statistically aligned, which in turn leads to an optimal spatial geometric registration. PR-Net achieves robust and superior performance for non-rigid registration of point sets, even in presence of Gaussian noise, outliers, and missing points, but requires much less time for registering large number of pairs. More importantly, for a new pair of point sets, PR-Net is able to directly predict the desired transformation using the learned model without repetitive iterative optimization routine. Our code is available at https://github.com/Lingjing324/PR-Net.

Motivation & Objective

• Motivate fast, real-time non-rigid point set registration for large datasets.
• Propose a learning-based framework that transfers registration patterns across pairs.
• Develop shape descriptor and correlation representations suitable for irregular point clouds.
• Demonstrate robustness to noise, outliers, and incomplete data while reducing computation time.

Proposed method

• Introduce shape descriptor tensors learned on reference grids over point sets to capture local/global geometry.
• Define a shape correlation tensor as all-to-all point-wise correlation between source and target descriptors.
• Use 2D/3D CNNs to map the shape correlation tensor to transformation parameters (rigid/non-rigid via TPS-based model).
• Adopt a statistical alignment loss by comparing the transformed source to the target via Chamfer Distance and Gaussian Mixture Model (GMM) likelihood.
• Train end-to-end with Adam optimizer, using a coarse-to-fine scheme by gradually annealing the GMM/σ parameter.
• Describe training settings including reference grids, MLP layer dimensions, and network architecture details.

Experimental results

Research questions

• RQ1 Can a learning-based model directly predict non-rigid transformations for unseen point sets without iterative optimization?
• RQ2 How can point clouds’ irregular structure be effectively represented for learning registration patterns?
• RQ3 What representations (shape descriptor and correlation tensors) enable robust alignment under noise, outliers, and incompleteness?
• RQ4 How does PR-Net compare to traditional non-learning registration methods in speed and accuracy?

Key findings

• PR-Net can directly predict geometric transformations for new point set pairs after training, avoiding per-pair optimization.
• On 10k 2D fish point-set pairs, PR-Net trains in ~13 minutes and tests in ~8 seconds, with Chamfer Distance similar to CPD (0.0037 training, 0.0044 testing, CPD ~0.0038).
• PR-Net maintains low Chamfer Distance (CD) for deformation levels up to ~1.0 (e.g., CD ≈ 0.015–0.016 for 2D fish at deformation 0.3–1.0).
• PR-Net shows robustness to data incompleteness, point drift, and outliers with CD remaining low until higher noise/omission levels (e.g., D.I. up to 0.6, D.O. up to 0.3).
• Compared to the CPD baseline, PR-Net offers substantially faster registration (seconds vs hours) and more stable generalization (lower CD std).
• The method generalizes across various 2D/3D shapes (hand, skeleton, skull, face, cat) with consistent performance under moderate deformation.