[Paper Review] Deep Closest Point: Learning Representations for Point Cloud Registration

Yue Wang, Justin Solomon|arXiv (Cornell University)|May 8, 2019

3D Shape Modeling and Analysis53 references120 citations

TL;DR

Deep Closest Point (DCP) learns point cloud embeddings and uses an attention-based matching module plus a differentiable SVD to predict a rigid transformation, outperforming ICP and several baselines on ModelNet40.

ABSTRACT

Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and natural language processing. Our model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer, to approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer to extract the final rigid transformation. We train our model end-to-end on the ModelNet40 dataset and show in several settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR), and the recently-proposed learning-based method PointNetLK. Beyond providing a state-of-the-art registration technique, we evaluate the suitability of our learned features transferred to unseen objects. We also provide preliminary analysis of our learned model to help understand whether domain-specific and/or global features facilitate rigid registration.

Motivation & Objective

• Motivate robust rigid registration beyond classical ICP to avoid local optima.
• Develop a learning-based pipeline that predicts reliable correspondences between two point clouds.
• Evaluate end-to-end performance against ICP, Go-ICP, FGR, and PointNetLK on ModelNet40.
• Investigate the role of local vs global features and the transferability of learned embeddings to unseen objects.

Proposed method

• Embed point clouds into a common feature space using PointNet or DGCNN.
• Incorporate an attention-based module (Transformer) to fuse contextual information across clouds.
• Generate a soft, differentiable correspondence (pointer) between points via a softmax over embeddings.
• Recover the rigid transformation with a differentiable SVD layer from the soft correspondences.
• Train end-to-end on synthetic pairs with a SE(3) loss combining rotation and translation errors.
• Enable ablations to compare embedding choices, MLP vs SVD, and embedding dimensionality.

Experimental results

Research questions

• RQ1 Can learned per-point embeddings improve robustness and accuracy of 3D point cloud registration over classical ICP?
• RQ2 Does an attention-based co-contextual embedding improve point correspondences between two clouds?
• RQ3 Is a differentiable SVD layer advantageous for recovering the rigid transform from soft correspondences?
• RQ4 How do local (DGCNN) versus global (PointNet) features affect registration performance and generalization?
• RQ5 Do the learned features transfer to unseen categories and maintain robustness to noise?

Key findings

• DCP-v1 already outperforms ICP, Go-ICP, FGR, and PointNetLK on the ModelNet40 test with unseen data.
• DCP-v2 with attention further improves registration accuracy over DCP-v1.
• DCP maintains robustness to Gaussian noise, outperforming competitors in noisy settings.
• Using DCP as an initialization step can enable ICP to converge to the global optimum, acting as a polishing stage.
• Ablations show that DGCNN-based local features and the SVD-based rigid motion estimator contribute to performance gains, while embedding dimension and architecture choices affect results.