[Paper Review] Correspondence Learning via Linearly-invariant Embedding

Riccardo Marin, Marie‐Julie Rakotosaona+2 more|arXiv (Cornell University)|Oct 25, 2020

3D Shape Modeling and Analysis57 references4 citations

TL;DR

This paper proposes a fully differentiable, end-to-end trainable framework for dense 3D point cloud correspondence learning by replacing fixed Laplace-Beltrami eigenfunctions with a learnable, linearly-invariant embedding. The method jointly optimizes a data-driven basis and optimal descriptor functions, achieving state-of-the-art accuracy in non-rigid correspondence tasks by leveraging learned embeddings with extra linear degrees of freedom for improved robustness.

ABSTRACT

In this paper, we propose a fully differentiable pipeline for estimating accurate dense correspondences between 3D point clouds. The proposed pipeline is an extension and a generalization of the functional maps framework. However, instead of using the Laplace-Beltrami eigenfunctions as done in virtually all previous works in this domain, we demonstrate that learning the basis from data can both improve robustness and lead to better accuracy in challenging settings. We interpret the basis as a learned embedding into a higher dimensional space. Following the functional map paradigm the optimal transformation in this embedding space must be linear and we propose a separate architecture aimed at estimating the transformation by learning optimal descriptor functions. This leads to the first end-to-end trainable functional map-based correspondence approach in which both the basis and the descriptors are learned from data. Interestingly, we also observe that learning a \emph{canonical} embedding leads to worse results, suggesting that leaving an extra linear degree of freedom to the embedding network gives it more robustness, thereby also shedding light onto the success of previous methods. Finally, we demonstrate that our approach achieves state-of-the-art results in challenging non-rigid 3D point cloud correspondence applications.

Motivation & Objective

• To address the limitations of fixed spectral bases in functional map-based 3D correspondence learning.
• To improve robustness and accuracy in non-rigid 3D shape correspondence under challenging deformations.
• To develop an end-to-end trainable pipeline where both the embedding basis and descriptor functions are learned from data.
• To investigate whether learning a canonical embedding is beneficial or if retaining linear degrees of freedom enhances performance.

Proposed method

• The method replaces traditional Laplace-Beltrami eigenfunctions with a differentiable, data-driven embedding network that learns a basis for representing shape functions.
• The learned embedding maps shapes into a higher-dimensional space where the optimal functional map is constrained to be linear.
• A separate neural network is trained to learn optimal descriptor functions that enable accurate correspondence estimation in the embedding space.
• The entire pipeline is end-to-end differentiable, allowing joint optimization of the embedding and descriptor networks via backpropagation.
• The framework preserves the functional map paradigm but generalizes it by learning the basis rather than relying on fixed spectral decomposition.
• The method explicitly retains an extra linear degree of freedom in the embedding, which is shown to improve robustness.

Experimental results

Research questions

• RQ1 Does learning a shape-specific embedding instead of using fixed Laplace-Beltrami eigenfunctions improve correspondence accuracy in non-rigid 3D shape matching?
• RQ2 Can an end-to-end trainable functional map framework be constructed where both the basis and descriptors are learned from data?
• RQ3 Is a canonical embedding (i.e., fixed basis) less robust than a basis with additional linear freedom in correspondence learning?
• RQ4 How does the proposed method compare to state-of-the-art approaches on challenging non-rigid 3D correspondence benchmarks?
• RQ5 What is the impact of the linear invariance property on the stability and generalization of learned correspondences?

Key findings

• The proposed method achieves state-of-the-art performance on non-rigid 3D point cloud correspondence benchmarks, outperforming previous functional map-based approaches.
• Learning a non-canonical, linearly-invariant embedding leads to better robustness and accuracy than learning a canonical embedding.
• The end-to-end differentiable training scheme enables joint optimization of both the embedding basis and descriptor functions, improving correspondence quality.
• The method demonstrates superior generalization on challenging non-rigid shapes with large deformations, where fixed spectral bases fail.
• The ablation study confirms that retaining an extra linear degree of freedom in the embedding space significantly improves performance.
• The results validate that data-driven basis learning enhances the functional map framework beyond the limitations of spectral methods.