[Paper Review] PPFNet: Global Context Aware Local Features for Robust 3D Point Matching

Haowen Deng, Tolga Birdal+1 more|arXiv (Cornell University)|Feb 7, 2018

Robotics and Sensor-Based Localization41 references56 citations

TL;DR

PPFNet learns global-context-aware local 3D descriptors for point cloud matching by combining point, normal, and point-pair features, using an N-tuple loss to inject global context and achieve state-of-the-art recall and robustness.

ABSTRACT

We present PPFNet - Point Pair Feature NETwork for deeply learning a globally informed 3D local feature descriptor to find correspondences in unorganized point clouds. PPFNet learns local descriptors on pure geometry and is highly aware of the global context, an important cue in deep learning. Our 3D representation is computed as a collection of point-pair-features combined with the points and normals within a local vicinity. Our permutation invariant network design is inspired by PointNet and sets PPFNet to be ordering-free. As opposed to voxelization, our method is able to consume raw point clouds to exploit the full sparsity. PPFNet uses a novel $ extit{N-tuple}$ loss and architecture injecting the global information naturally into the local descriptor. It shows that context awareness also boosts the local feature representation. Qualitative and quantitative evaluations of our network suggest increased recall, improved robustness and invariance as well as a vital step in the 3D descriptor extraction performance.

Motivation & Objective

• Motivate robust local 3D descriptors that are repeatable under partial/noisy inputs.
• Develop a permutation-invariant network that can operate on raw, unstructured point clouds.
• Incorporate global contextual information into local descriptors to improve matching performance.
• Propose an N-tuple loss to capture many-to-many correspondences and guide end-to-end learning.
• Demonstrate state-of-the-art recall, speed, and robustness on real 3D datasets.

Proposed method

• Represent local geometry as a set consisting of points, normals, and point-pair-features (PPF).
• Extend PointNet with a multi-patch, global-context fusion where local patch features are max-pooled to a global context and concatenated back to locals.
• Introduce N-tuple loss, a generalized N-to-N contrastive loss, that uses a correspondence matrix and a feature-distance matrix to encourage correct patch correspondences across fragments.
• Train on correspondences between 3D fragments with random fragment pairs to learn globally informed descriptors.
• Evaluate descriptors with a RANSAC-based pipeline and measure recall on real 3DMatch, SUN3D, 7-scenes datasets.

Experimental results

Research questions

• RQ1 Can global context be effectively injected into local 3D descriptors to improve correspondences under partial or noisy data?
• RQ2 Does incorporating point-pair features alongside raw points and normals improve discriminativity and rotation robustness?
• RQ3 Is an N-tuple loss more effective than traditional pair/triplet losses for many-to-many patch correspondences in 3D matching?
• RQ4 How does PPFNet perform relative to hand-crafted and other learning-based descriptors in recall, robustness to sparsity, and speed?
• RQ5 What is the impact of including global context versus using vanilla PointNet for local feature learning?

Key findings

• PPFNet outperforms hand-crafted baselines in mean recall on the 3DMatch benchmark before RANSAC.
• PPFNet shows robustness to decreasing point density, maintaining higher recall as sparsity increases.
• Adding global context to local features improves matching by 18% in training and 7% in validation over a vanilla PointNet baseline.
• Incorporating point-pair features yields a measurable performance gain over not using PPF inputs.
• PPFNet achieves faster inference and a landmark recall advantage over 3DMatch with similar patch counts.