[Paper Review] GeoTransformer: Fast and Robust Point Cloud Registration with Geometric Transformer

Zheng Qin, Hao Yu+6 more|arXiv (Cornell University)|Jul 25, 2023

3D Surveying and Cultural HeritageEarth and Planetary Sciences3 citations

TL;DR

GeoTransformer proposes a geometrically invariant transformer architecture that encodes pairwise distances and triplet-wise angles between superpoints to achieve robust, RANSAC-free point cloud registration. By leveraging transformation-invariant geometric features, it achieves 100× faster registration and state-of-the-art performance on low-overlap benchmarks like 3DLoMatch, improving inlier ratio by 18–31 percentage points and registration recall by over 7 points.

ABSTRACT

We study the problem of extracting accurate correspondences for point cloud registration. Recent keypoint-free methods have shown great potential through bypassing the detection of repeatable keypoints which is difficult to do especially in low-overlap scenarios. They seek correspondences over downsampled superpoints, which are then propagated to dense points. Superpoints are matched based on whether their neighboring patches overlap. Such sparse and loose matching requires contextual features capturing the geometric structure of the point clouds. We propose Geometric Transformer, or GeoTransformer for short, to learn geometric feature for robust superpoint matching. It encodes pair-wise distances and triplet-wise angles, making it invariant to rigid transformation and robust in low-overlap cases. The simplistic design attains surprisingly high matching accuracy such that no RANSAC is required in the estimation of alignment transformation, leading to $100$ times acceleration. Extensive experiments on rich benchmarks encompassing indoor, outdoor, synthetic, multiway and non-rigid demonstrate the efficacy of GeoTransformer. Notably, our method improves the inlier ratio by $18{\sim}31$ percentage points and the registration recall by over $7$ points on the challenging 3DLoMatch benchmark. Our code and models are available at \url{https://github.com/qinzheng93/GeoTransformer}.

Motivation & Objective

• To address the challenge of low-overlap point cloud registration where keypoint detection fails due to poor repeatability.
• To improve superpoint matching accuracy by encoding geometric structure invariant to rigid transformations.
• To eliminate reliance on RANSAC for transformation estimation by enabling high-quality, inlier-rich correspondences from the start.
• To accelerate registration by replacing iterative robust estimation with a direct, differentiable alignment step.
• To achieve state-of-the-art performance across diverse benchmarks, including indoor, outdoor, synthetic, multiway, and non-rigid settings.

Proposed method

• GeoTransformer uses a self-attention mechanism that attends to all other superpoints based on pair-wise distances and triplet-wise angles, ensuring geometric invariance.
• It encodes geometric consistency through distance and angle features, making the representation invariant to rigid transformations.
• An overlap-aware circle loss is introduced to focus training on superpoint pairs with higher patch overlap, improving robustness in low-overlap regimes.
• Dense correspondences are predicted via an optimal transport layer using the high-quality superpoint matches, avoiding RANSAC.
• The final alignment is estimated directly from the dense correspondences using a differentiable, parameter-free LGR (Low-rank Global Registration) module.
• The method operates in a coarse-to-fine manner: superpoint matching is performed first, then propagated to dense points.

Experimental results

Research questions

• RQ1 Can a transformer-based method learn geometrically invariant features that improve superpoint matching in low-overlap scenarios?
• RQ2 Can geometric invariance be achieved without relying on coordinate-based positional embeddings that are transformation-variant?
• RQ3 Can high-accuracy superpoint matching eliminate the need for RANSAC in point cloud registration?
• RQ4 How does GeoTransformer perform on challenging benchmarks with large rotations and minimal overlap?
• RQ5 Can the method scale to multiway and non-rigid registration tasks while maintaining robustness and speed?

Key findings

• GeoTransformer improves the inlier ratio by 18–31 percentage points and registration recall by over 7 points on the 3DLoMatch benchmark, significantly outperforming prior methods.
• The method achieves 100× speedup in registration by eliminating RANSAC, with inference time reduced to 0.01 seconds for 5,000 correspondences.
• On the 3DMatch benchmark, GeoTransformer achieves 95.7% registration recall with RTE of 5.69 cm and RRE of 1.92°, outperforming all baselines including PointDSC and DHVR.
• On the KITTI odometry benchmark, it achieves 99.5% registration recall with RTE of 6.5 cm and RRE of 0.24°, demonstrating strong robustness under large rotations and real-world noise.
• The dual-normalization operation in the superpoint matching module slightly improves performance in low-overlap cases by mitigating outlier matches from geometrically ambiguous patches.
• Pose refinement with 5 iterations in LGR achieves near-saturation in accuracy, balancing speed and precision for real-time deployment.