[Paper Review] PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency

Xuyang Bai, Zixin Luo+6 more|arXiv (Cornell University)|Mar 9, 2021

3D Surveying and Cultural HeritageEarth and Planetary Sciences76 references17 citations

TL;DR

PointDSC proposes a deep learning framework for robust 3D point cloud registration by explicitly modeling spatial consistency through a novel nonlocal feature aggregation module and a differentiable spectral matching module. It achieves state-of-the-art performance on real-world datasets, significantly outperforming existing learning-based and hand-crafted outlier rejection methods, especially under high outlier ratios and small overlaps.

ABSTRACT

Removing outlier correspondences is one of the critical steps for successful feature-based point cloud registration. Despite the increasing popularity of introducing deep learning methods in this field, spatial consistency, which is essentially established by a Euclidean transformation between point clouds, has received almost no individual attention in existing learning frameworks. In this paper, we present PointDSC, a novel deep neural network that explicitly incorporates spatial consistency for pruning outlier correspondences. First, we propose a nonlocal feature aggregation module, weighted by both feature and spatial coherence, for feature embedding of the input correspondences. Second, we formulate a differentiable spectral matching module, supervised by pairwise spatial compatibility, to estimate the inlier confidence of each correspondence from the embedded features. With modest computation cost, our method outperforms the state-of-the-art hand-crafted and learning-based outlier rejection approaches on several real-world datasets by a significant margin. We also show its wide applicability by combining PointDSC with different 3D local descriptors.

Motivation & Objective

• To address the challenge of high outlier ratios in feature-based 3D point cloud registration, where traditional methods fail due to poor inlier cluster dominance.
• To overcome the limitation of existing learning-based methods that ignore spatial consistency during feature embedding and outlier classification.
• To integrate geometric constraints—specifically, spatial consistency under rigid transformations—into deep neural networks for more robust outlier rejection.
• To develop a method that generalizes across indoor and outdoor scenarios and works seamlessly with various 3D local descriptors.
• To improve the robustness of spectral matching in high-noise regimes by incorporating learned geometric features and a seeding mechanism.

Proposed method

• Proposes a spatial-consistency guided nonlocal (SCNonlocal) module that weighs feature correlation using both feature similarity and spatial coherence (length consistency under rigid transformation).
• Introduces a differentiable neural spectral matching (NSM) module that replaces traditional length-based compatibility with deep geometric features, enabling end-to-end training.
• Employs a seeding mechanism that identifies reliable correspondences and forms multiple subsets for iterative NSM, improving inlier cluster detection under high outlier ratios.
• Uses a compatibility graph constructed from embedded features and spatial consistency to perform spectral clustering for inlier set estimation.
• Supervises the NSM module via a pairwise spatial compatibility loss ($L_{sm}$), enabling the network to learn inlier confidence from geometric consistency.
• Combines the SCNonlocal and NSM modules in a two-stage pipeline: first, feature embedding with spatial-aware aggregation; second, differentiable outlier pruning via spectral analysis.

Experimental results

Research questions

• RQ1 Can explicit modeling of spatial consistency in deep networks improve outlier rejection robustness in 3D point cloud registration?
• RQ2 How does incorporating geometric constraints (rigid transformation consistency) into feature embedding enhance representation learning for correspondence matching?
• RQ3 To what extent can a differentiable spectral matching module outperform traditional SM and end-to-end learning-based methods under high outlier ratios?
• RQ4 Does the proposed seeding mechanism improve performance on small-overlap and high-noise registration scenarios?
• RQ5 Can the method generalize across diverse datasets and work effectively with various 3D local descriptors?

Key findings

• PointDSC achieves 93.28% registration recall (RR) and 82.35% F1-score on the 3DMatch dataset, outperforming the previous SOTA (DGR) by 5.1% in RR and 5.2% in F1.
• The SCNonlocal module improves registration performance by 4.89% in RR and 7.31% in F1 compared to the baseline PointCN when combined with the NSM module.
• The NSM module with neural feature consistency boosts traditional SM by 1.86% in RR and 0.67% in F1, demonstrating the benefit of learned geometric features.
• The seeding mechanism increases performance by 4.48% in RR and 33.47% in F1 compared to non-seeded NSM, proving its effectiveness in high-outlier regimes.
• Removing the $L_{sm}$ loss during training reduces F1-score to 81.58%, confirming the importance of supervision via spatial compatibility.
• Qualitative results show PointDSC successfully rejects outliers even at 95% outlier ratios, while RANSAC-100k fails in such extreme cases.