[Paper Review] Lepard: Learning partial point cloud matching in rigid and deformable scenes

Yang Li, Tatsuya Harada|arXiv (Cornell University)|Nov 24, 2021

3D Shape Modeling and Analysis4 citations

TL;DR

Lepard proposes a learning-based method for partial point cloud matching in both rigid and deformable scenes by disentangling feature and 3D position representations, using dot-product-based position encoding and a repositioning module to enhance geometric awareness. It achieves state-of-the-art registration recall of 93.9% on 3DMatch and 71.3% on 3DLoMatch, and +27.1% and +34.8% higher non-rigid matching recall on the new 4DMatch and 4DLoMatch benchmarks.

ABSTRACT

We present Lepard, a Learning based approach for partial point cloud matching in rigid and deformable scenes. The key characteristics are the following techniques that exploit 3D positional knowledge for point cloud matching: 1) An architecture that disentangles point cloud representation into feature space and 3D position space. 2) A position encoding method that explicitly reveals 3D relative distance information through the dot product of vectors. 3) A repositioning technique that modifies the crosspoint-cloud relative positions. Ablation studies demonstrate the effectiveness of the above techniques. In rigid cases, Lepard combined with RANSAC and ICP demonstrates state-of-the-art registration recall of 93.9% / 71.3% on the 3DMatch / 3DLoMatch. In deformable cases, Lepard achieves +27.1% / +34.8% higher non-rigid feature matching recall than the prior art on our newly constructed 4DMatch / 4DLoMatch benchmark.

Motivation & Objective

• Address the challenge of ambiguous feature matching in scenes with repetitive geometry patterns by incorporating 3D positional knowledge.
• Improve robustness in partial and low-overlap point cloud matching, especially in deformable scenes with occlusions and non-rigid deformations.
• Develop a unified framework that performs well on both rigid and non-rigid point cloud matching without requiring separate architectures.
• Introduce a new benchmark, 4DMatch, and its low-overlap variant 4DLoMatch, to evaluate non-rigid point cloud matching under realistic dynamic conditions.
• Overcome limitations of translation-invariant deep networks by explicitly modeling relative 3D positions through learnable, differentiable components.

Proposed method

• Disentangle point cloud representation into separate feature space and 3D position space to decouple geometric features from spatial context.
• Introduce a position encoding method that computes relative 3D distance information via the dot product of 3D vector representations.
• Apply a repositioning module that adjusts cross-point-cloud relative positions to improve attention and differentiable matching performance.
• Use a Transformer-based architecture with self- and cross-attention mechanisms to aggregate context and predict correspondences.
• Leverage differentiable matching with Sinkhorn normalization to enable end-to-end optimization of correspondence predictions.
• Integrate predictions into RANSAC and ICP for rigid registration, and into non-rigid ICP (N-ICP) for deformable registration.

Experimental results

Research questions

• RQ1 Can explicit modeling of 3D relative positional information improve feature matching in partial and overlapping point clouds?
• RQ2 How does disentangling feature and position representations affect matching accuracy in rigid and deformable scenes?
• RQ3 To what extent can a unified architecture outperform prior methods on both rigid and non-rigid point cloud matching benchmarks?
• RQ4 How does the proposed method perform under low overlap and occlusion conditions compared to state-of-the-art approaches?
• RQ5 Can a learnable repositioning module enhance cross-attention and differentiable matching in point cloud correspondence learning?

Key findings

• Lepard achieves a registration recall of 93.9% on 3DMatch and 71.3% on 3DLoMatch when combined with RANSAC and ICP, outperforming prior methods.
• On the new 4DMatch benchmark, Lepard achieves a non-rigid feature matching recall that is 27.1% higher than the prior art.
• On the 4DLoMatch benchmark, Lepard improves non-rigid matching recall by 34.8% compared to the state of the art.
• Ablation studies confirm that the repositioning module and position encoding method significantly improve performance, especially in low-overlap scenarios.
• The method outperforms Predator in both RRE and RTE metrics, with 1.96° and 6.0 cm on 3DMatch, respectively, after ICP refinement.
• Failure cases in low-overlap settings persist when non-overlapping regions contain similar geometric patterns, indicating a challenge for future work.