[Paper Review] PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling

Yan Xu, Chaoda Zheng+3 more|arXiv (Cornell University)|Mar 1, 2020

3D Shape Modeling and Analysis50 references49 citations

TL;DR

PointASNL introduces adaptive sampling and a local-nonlocal module to robustly process noisy/unstructured point clouds, achieving state-of-the-art results on classification and segmentation across synthetic indoor/outdoor datasets.

ABSTRACT

Raw point clouds data inevitably contains outliers or noise through acquisition from 3D sensors or reconstruction algorithms. In this paper, we present a novel end-to-end network for robust point clouds processing, named PointASNL, which can deal with point clouds with noise effectively. The key component in our approach is the adaptive sampling (AS) module. It first re-weights the neighbors around the initial sampled points from farthest point sampling (FPS), and then adaptively adjusts the sampled points beyond the entire point cloud. Our AS module can not only benefit the feature learning of point clouds, but also ease the biased effect of outliers. To further capture the neighbor and long-range dependencies of the sampled point, we proposed a local-nonlocal (L-NL) module inspired by the nonlocal operation. Such L-NL module enables the learning process insensitive to noise. Extensive experiments verify the robustness and superiority of our approach in point clouds processing tasks regardless of synthesis data, indoor data, and outdoor data with or without noise. Specifically, PointASNL achieves state-of-the-art robust performance for classification and segmentation tasks on all datasets, and significantly outperforms previous methods on real-world outdoor SemanticKITTI dataset with considerate noise. Our code is released through https://github.com/yanx27/PointASNL.

Motivation & Objective

• Address robustness of point cloud processing to noise and outliers from real-world sensors.
• Develop an end-to-end architecture that adapts sampled points to intrinsic geometry.
• Capture both local neighborhood information and long-range dependencies in point clouds.
• Demonstrate improved performance on diverse datasets (synthetic, indoor, outdoor).

Proposed method

• Adaptive Sampling (AS) module that re-weights and shifts sampled points beyond FPS to fit intrinsic geometry and reduce outlier impact.
• Local-Nonlocal (L-NL) module combining a Point Local (PL) cell and a Point Nonlocal (PNL) cell to fuse local and global context.
• Point Nonlocal cell computes responses for sampled points as weighted sums over all points to encode long-range dependencies.
• Pluggable PL and PNL cells enable flexible local/global feature learning within the L-NL framework.
• Hierarchical PointASNL architecture integrates AS and L-NL across layers for classification and segmentation.
• Soft-attention based feature weighting and differentiable sampling enable end-to-end training.

Experimental results

Research questions

• RQ1 How can adaptive sampling improve robustness of point cloud processing in the presence of noise/outliers?
• RQ2 Can a nonlocal, multi-scale context module improve learning by capturing long-range dependencies in point clouds?
• RQ3 Does combining adaptive sampling with local and nonlocal feature aggregation yield superior classification and segmentation performance on diverse datasets?
• RQ4 How does PointASNL perform relative to state-of-the-art methods on synthetic and real-world outdoor datasets under noisy conditions?

Key findings

• PointASNL achieves state-of-the-art robustness and accuracy in classification and segmentation across synthetic, indoor, and outdoor datasets.
• Adaptive sampling reduces sensitivity to outliers and allows sampled points to better reflect intrinsic geometry.
• The local-nonlocal module effectively fuses local neighborhood information with global context, improving learning under noise.
• PNL enables long-range dependency modeling by attending over the entire point set, enhancing robustness to sparse/noisy data.
• Ablation studies show substantial gains when combining PL, PNL, and AS versus using local or global features alone.
• On outdoor SemanticKITTI, PointASNL significantly outperforms competing methods in segmentation under noisy conditions.