[Paper Review] RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut $2^D$-Tree Representation

Sk Aziz Ali, Kerem Kahraman+2 more|arXiv (Cornell University)|Apr 12, 2021

Robotics and Sensor-Based Localization4 citations

TL;DR

RPSRNet is an end-to-end trainable deep neural network for rigid point set registration that uses a Barnes-Hut $2^D$-tree representation to encode input point clouds and employs hierarchical feature embedding with iterative transformation refinement. It achieves state-of-the-art accuracy and inference speeds of ~12–15 ms per 250K-point cloud pair, outperforming methods like DCP-v2 and PointNetLK on KITTI and ModelNet40 datasets, especially under noise and non-uniform sampling.

ABSTRACT

We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state-of-the-art methods - e.g., on the KITTI data set, DCP-v2 by1.3 and 1.5 times, and PointNetLK by 1.8 and 1.9 times better rotational and translational accuracy respectively. Evaluation on ModelNet40 shows that RPSRNet is more robust than other benchmark methods when the samples contain a significant amount of noise and other disturbances. RPSRNet accurately registers point clouds with non-uniform sampling densities, e.g., LiDAR data, which cannot be processed by many existing deep-learning-based registration methods.

Motivation & Objective

• To develop a deep learning-based method for rigid point set registration that is both accurate and efficient for large-scale, real-world point clouds.
• To address the limitations of existing deep learning methods in handling non-uniformly sampled point clouds, such as LiDAR data.
• To improve robustness against noise and partial overlaps in point cloud data.
• To enable end-to-end training with high inference speed suitable for real-time applications.
• To introduce a novel $2^D$-tree representation that supports hierarchical, differentiable feature learning for point clouds.

Proposed method

• The method uses Barnes-Hut $2^D$-trees to represent input point clouds, encoding center-of-mass and point-density per node as input features.
• A hierarchical, tree-convolutional neural network processes the $2^D$-tree structure to learn global geometric features through multi-level aggregation.
• An iterative transformation refinement module updates the estimated rigid transformation at intermediate stages using predicted feature correspondences.
• The network regresses the rotation matrix $\mathbf{R} \in \mathcal{SO}(D)$ and translation vector $\mathbf{t} \in \mathbb{R}^D$ from the final feature representation.
• The model is trained end-to-end using a differentiable loss function that minimizes the difference between predicted and ground-truth transformations.
• The $2^D$-tree construction is performed on CPU before training, with node attributes passed as input tensors to the network.

Experimental results

Research questions

• RQ1 Can a learnable $2^D$-tree representation improve the accuracy and efficiency of deep learning-based rigid point set registration?
• RQ2 Does iterative refinement of the transformation estimate enhance feature matching accuracy in a deep neural network framework?
• RQ3 How does RPSRNet perform on large-scale, non-uniformly sampled LiDAR point clouds compared to state-of-the-art methods?
• RQ4 Can the network maintain high accuracy under significant noise and partial overlap, common in real-world scans?
• RQ5 Is the proposed method significantly faster than existing deep learning-based registration networks while maintaining or improving accuracy?

Key findings

• On the KITTI LiDAR odometry dataset, RPSRNet achieves 1.3× and 1.5× better rotational and translational accuracy, respectively, compared to DCP-v2.
• On the same dataset, RPSRNet improves upon PointNetLK by 1.8× and 1.9× in rotational and translational accuracy, respectively.
• On ModelNet40 with 40% Gaussian noise, RPSRNet maintains high accuracy, outperforming other methods that diverge under similar conditions.
• RPSRNet achieves inference speeds of ~12–15 ms per 250K-point cloud pair, making it the fastest among competing deep learning-based methods.
• The ablation study confirms that iterative refinement in RPSRNet stabilizes learning, while DCP-v2 variants diverge under the same settings, highlighting the superiority of the proposed architecture and representation.
• The method successfully registers point clouds with non-uniform sampling densities, a capability lacking in many existing deep learning-based registration methods.