[Paper Review] Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction

Guangyan Chen, Meiling Wang+3 more|arXiv (Cornell University)|Dec 17, 2021

3D Surveying and Cultural Heritage4 citations

TL;DR

This paper proposes the Deep Interaction Transformer (DIT), a full Transformer framework for robust point cloud registration that enhances feature discrimination and outlier rejection through global structure modeling, deep information interaction via positional encoding, and geometric consistency-based confidence evaluation. DIT achieves 94.7% success rate on noisy partial-to-partial registration, outperforming state-of-the-art methods.

ABSTRACT

Recent Transformer-based methods have achieved advanced performance in point cloud registration by utilizing advantages of the Transformer in order-invariance and modeling dependency to aggregate information. However, they still suffer from indistinct feature extraction, sensitivity to noise, and outliers. The reasons are: (1) the adoption of CNNs fails to model global relations due to their local receptive fields, resulting in extracted features susceptible to noise; (2) the shallow-wide architecture of Transformers and lack of positional encoding lead to indistinct feature extraction due to inefficient information interaction; (3) the omission of geometrical compatibility leads to inaccurate classification between inliers and outliers. To address above limitations, a novel full Transformer network for point cloud registration is proposed, named the Deep Interaction Transformer (DIT), which incorporates: (1) a Point Cloud Structure Extractor (PSE) to model global relations and retrieve structural information with Transformer encoders; (2) a deep-narrow Point Feature Transformer (PFT) to facilitate deep information interaction across two point clouds with positional encoding, such that Transformers can establish comprehensive associations and directly learn relative position between points; (3) a Geometric Matching-based Correspondence Confidence Evaluation (GMCCE) method to measure spatial consistency and estimate inlier confidence by designing the triangulated descriptor. Extensive experiments on clean, noisy, partially overlapping point cloud registration demonstrate that our method outperforms state-of-the-art methods.

Motivation & Objective

• To address the limitations of existing Transformer-based methods in point cloud registration, including poor noise robustness, indistinct feature extraction, and inaccurate outlier detection.
• To overcome the reliance on CNNs for local feature extraction, which limits global context modeling and increases sensitivity to noise.
• To improve feature discrimination and robustness by enabling deep, narrow Transformer architectures with positional encoding for relative position learning.
• To enhance inlier-outlier classification accuracy by incorporating geometric consistency through a triangulated descriptor-based confidence evaluation.
• To develop a systematic full Transformer framework that unifies structural modeling, deep feature interaction, and geometric matching for superior registration performance.

Proposed method

• The Point Cloud Structure Extractor (PSE) uses Transformer encoders to model long-range dependencies across the entire point cloud and incorporates a Local Feature Integrator (LFI) to enhance structural feature extraction.
• The Point Feature Transformer (PFT) employs a deep-narrow architecture with a dedicated positional encoding network to enable the model to learn relative spatial positions between points, facilitating comprehensive cross-point cloud information interaction.
• A novel Geometric Matching-based Correspondence Confidence Evaluation (GMCCE) method is introduced, using a rotation-invariant triangulated descriptor to measure spatial consistency and estimate inlier confidence.
• The PSE and PFT modules are jointly trained to extract robust, discriminative features by integrating global structural context and fine-grained relative position information.
• The full DIT framework integrates PSE, PFT, and GMCCE into an end-to-end trainable system that improves feature representation and alignment accuracy.
• The model is trained and evaluated on ModelNet40 with extensive ablation studies to validate each component’s contribution to robustness and accuracy.

Experimental results

Research questions

• RQ1 Can a full Transformer architecture outperform hybrid CNN-Transformer models in point cloud registration by enabling deeper and more effective information interaction?
• RQ2 How does incorporating positional encoding in a deep-narrow Transformer architecture improve feature discrimination and robustness to noise in point cloud registration?
• RQ3 To what extent can geometric consistency, measured via a triangulated descriptor, improve inlier confidence estimation and outlier rejection in registration tasks?
• RQ4 Does modeling global relations through a dedicated structure extractor enhance robustness to partial overlap and high noise levels in point clouds?
• RQ5 What is the individual contribution of PSE, PFT, and GMCCE to overall registration accuracy and success rate in challenging settings?

Key findings

• DIT achieves a 94.7% success rate on high-noise partial-to-partial point cloud registration, significantly outperforming all baselines.
• The ablation study shows that removing the PSE module reduces success rate to 1.2%, demonstrating its critical role in modeling global structure and noise robustness.
• Excluding the positional encoding network leads to a 39.4% drop in success rate, confirming its importance for learning relative spatial positions.
• Removing the GMCCE module results in a 40–74% drop in accuracy, highlighting its effectiveness in distinguishing inliers from outliers using geometric consistency.
• The deep-narrow PFT architecture improves performance by 55–80% compared to a shallow-wide variant, proving the benefit of deeper information interaction.
• On the most challenging high-noise partial-to-partial setting, DIT achieves 98% success rate in both rotation and translation, outperforming RGM (94%) and all other methods (≤80%)