[Paper Review] FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling

Kangcheng Liu, Zhi Gao+2 more|arXiv (Cornell University)|Dec 17, 2020

3D Shape Modeling and Analysis63 references32 citations

TL;DR

FG-Net is a general deep learning framework for fast, accurate understanding of large-scale LiDAR point clouds without voxelization, achieving real-time performance on a GTX 1080.

ABSTRACT

This work presents FG-Net, a general deep learning framework for large-scale point clouds understanding without voxelizations, which achieves accurate and real-time performance with a single NVIDIA GTX 1080 GPU. First, a novel noise and outlier filtering method is designed to facilitate subsequent high-level tasks. For effective understanding purpose, we propose a deep convolutional neural network leveraging correlated feature mining and deformable convolution based geometric-aware modelling, in which the local feature relationships and geometric patterns can be fully exploited. For the efficiency issue, we put forward an inverse density sampling operation and a feature pyramid based residual learning strategy to save the computational cost and memory consumption respectively. Extensive experiments on real-world challenging datasets demonstrated that our approaches outperform state-of-the-art approaches in terms of accuracy and efficiency. Moreover, weakly supervised transfer learning is also conducted to demonstrate the generalization capacity of our method.

Motivation & Objective

• Address noise and outliers in LiDAR point clouds to improve downstream understanding tasks.
• Develop a point-based network that leverages correlated feature mining and deformable geometry modelling for large-scale scenes.
• Enhance efficiency through inverse density sampling and a feature pyramid residual architecture.
• Achieve real-time inference on large-scale point clouds and demonstrate generalization via weakly supervised transfer learning.

Proposed method

• Introduce Radius and Gaussian Distribution Based Noise and Outliers Filtering to pre-clean point clouds.
• Propose FG-Conv module with three subcomponents: Pointwise Correlated Features Mining (PFM), Geometric Convolutional Modelling (GCM), and Attentional Aggregation (AG).
• Use deformable 3D kernels to adapt to local geometric structures for robust local geometry modelling.
• Apply an attention mechanism to fuse local and non-local features across neighbours.
• Implement a feature pyramid residual learning framework to fuse multi-resolution features efficiently.
• Incorporate a global self-attention module to capture long-range dependencies and augment local features.

Experimental results

Research questions

• RQ1 How does FG-Net perform on large-scale LiDAR datasets in terms of accuracy for semantic segmentation and object classification?
• RQ2 Can FG-Net achieve real-time or near real-time inference on standard GPUs without voxelization?
• RQ3 Do the PFMs, GCMs, and attentional aggregation components provide measurable gains over baseline point-based methods?
• RQ4 How well does FG-Net generalize to unseen scenes via weakly supervised transfer learning?
• RQ5 What is the impact of the noise/outlier filtering and inverse density sampling on final performance?

Key findings

• FG-Net outperforms state-of-the-art methods in accuracy and efficiency on challenging real-world datasets.
• The model achieves real-time segmentation performance (example: 0.021s per 1e5 points) on a GTX 1080 GPU.
• A fast noise/outlier filtering method (0.61s per 1e6 points) improves robustness and downstream performance.
• Weakly supervised transfer learning demonstrates generalization to new scenes.
• A feature pyramid residual architecture enables memory-efficient multi-scale fusion without sacrificing accuracy.