[Paper Review] Deep Learning for 3D Point Clouds: A Survey

Yulan Guo, Hanyun Wang+4 more|arXiv (Cornell University)|Dec 27, 2019

3D Shape Modeling and Analysis252 references48 citations

TL;DR

A comprehensive survey of deep learning methods for 3D point clouds, covering 3D shape classification, object detection/tracking, and point cloud segmentation, with dataset comparisons and future directions.

ABSTRACT

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

Motivation & Objective

• Motivate and organize the study of deep learning methods for 3D point clouds.
• Provide a taxonomy and state-of-the-art overview for 3D shape classification, object detection/tracking, and segmentation.
• Compare methods on publicly available datasets and summarize evaluation metrics and trends.
• Highlight datasets and evaluation protocols to guide future research.

Proposed method

• Classify methods into multi-view, volumetric, and point-based approaches with emphasis on point-based methods.
• Survey various convolution frameworks for 3D point clouds, including continuous and discrete convolutions, and graph-based and hierarchical methods.
• Present a structured review of architectures such as PointNet/PointNet++ and successors, along with representative spatial, spectral, and hierarchical techniques.
• Summarize publicly available datasets and evaluation metrics used across tasks.

Experimental results

Research questions

• RQ1 What are the dominant deep learning paradigms for processing 3D point clouds and how do they differ in capturing geometry and structure?
• RQ2 How do 3D point cloud methods perform across key tasks (classification, detection/tracking, segmentation) on standard datasets?
• RQ3 What datasets and evaluation metrics are most informative for benchmarking deep learning methods on point clouds?
• RQ4 What are the main future directions and unresolved challenges in deep learning for point clouds?

Key findings

• The survey covers three major tasks: 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation.
• It provides comparative results on several publicly available datasets and discusses insights and future directions.
• Diverse data representations (points, meshes, RGB-D, etc.) and evaluation metrics (OA, mAcc, AP, mIoU, AMOTA, AMOTP) are summarized.
• The paper highlights the evolution from multi-view and volumetric methods to predominantly point-based approaches, with numerous architectures built on PointNet/PointNet++ and graph/convolution frameworks.
• A regularly updated project page with state-of-the-art results is provided (GitHub project: SoTA-Point-Cloud).