[Paper Review] Accurate Point Cloud Registration with Robust Optimal Transport

Zhengyang Shen, Jean Feydy+5 more|arXiv (Cornell University)|Nov 1, 2021

3D Shape Modeling and Analysis26 citations

TL;DR

The paper demonstrates scalable robust optimal transport (RobOT) as an effective feature-matching layer to improve point cloud registration, enabling state-of-the-art results on scene flow and lung registration while maintaining affordability. It introduces a compact RobOT representation and a Deep-RobOT architecture that pre-aligns, refines, and fine-tunes registrations.

ABSTRACT

This work investigates the use of robust optimal transport (OT) for shape matching. Specifically, we show that recent OT solvers improve both optimization-based and deep learning methods for point cloud registration, boosting accuracy at an affordable computational cost. This manuscript starts with a practical overview of modern OT theory. We then provide solutions to the main difficulties in using this framework for shape matching. Finally, we showcase the performance of transport-enhanced registration models on a wide range of challenging tasks: rigid registration for partial shapes; scene flow estimation on the Kitti dataset; and nonparametric registration of lung vascular trees between inspiration and expiration. Our OT-based methods achieve state-of-the-art results on Kitti and for the challenging lung registration task, both in terms of accuracy and scalability. We also release PVT1010, a new public dataset of 1,010 pairs of lung vascular trees with densely sampled points. This dataset provides a challenging use case for point cloud registration algorithms with highly complex shapes and deformations. Our work demonstrates that robust OT enables fast pre-alignment and fine-tuning for a wide range of registration models, thereby providing a new key method for the computer vision toolbox. Our code and dataset are available online at: https://github.com/uncbiag/robot.

Motivation & Objective

• Motivate and formalize robust optimal transport (RobOT) as a robust, scalable tool for point cloud registration and matching.
• Show how RobOT improves feature matching across rigid, affine, and nonparametric registrations in challenging scenarios.
• Demonstrate practical integration of RobOT with deformation models and deep learning for efficient, accurate registrations.
• Provide new benchmarks and datasets (lung vascular trees) to evaluate high-resolution registration pipelines.

Proposed method

• Present RobOT theory and a stable, memory-efficient representation of the transport plan via weighted RobOT matching (v_i and w_i).
• Use dual variables from OT solvers to compute v_i (displacement) and w_i (confidence) with O(N+M) memory.
• Integrate RobOT with regularization models to obtain rigid, affine, or spline-based deformations (Eqs. 5–7).
• Propose Deep-RobOT (D-RobOT): a three-stage pipeline (OT pre-alignment, deep deformation predictor, OT post-processing) for end-to-end registration.
• Employ unbalanced OT via KL penalties to handle partial overlap and sampling artifacts.
• Provide fast, differentiable RobOT layers enabling scaling to high-resolution point clouds (up to 100k points).

Experimental results

Research questions

• RQ1 Can robust optimal transport (RobOT) provide reliable, scalable correspondences for high-resolution 3D point clouds?
• RQ2 How does RobOT interact with regularization strategies and deformation models to improve registration accuracy?
• RQ3 What are the benefits of a Deep-RobOT architecture (pre-alignment, deep predictor, post-processing) for challenging tasks like scene flow and medical lung registration?
• RQ4 Is RobOT capable of handling partial overlap and outliers common in real-world data while maintaining efficiency?

Key findings

• RobOT achieves strong baseline results at high speed and low memory cost compared to many deep learning methods.
• RobOT-enhanced pipelines consistently outperform state-of-the-art methods on challenging tasks, especially at high resolutions.
• The three-step D-RobOT architecture (pre-align, deep registration, post-process) yields robust, accurate registrations across synthetic and real data.
• The proposed lung vascular tree dataset (PVT1010) provides a large, high-resolution benchmark with densely sampled points.
• RobOT enables effective fine-tuning of registrations, translating “good” matchings into near-perfect alignments at affordable cost.
• Unbalanced OT with KL penalties supports partial registrations and makes RobOT robust to sampling artifacts.