[Paper Review] Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation

Hui Zhou, Xinge Zhu+5 more|arXiv (Cornell University)|Aug 4, 2020

Advanced Neural Network Applications33 references138 citations

TL;DR

Cylinder3D demonstrates that processing LiDAR point clouds directly in 3D with a cylinder-based partition and 3D convolutions, plus specialized blocks, yields state-of-the-art driving-scene segmentation on SemanticKITTI.

ABSTRACT

State-of-the-art methods for large-scale driving-scene LiDAR semantic segmentation often project and process the point clouds in the 2D space. The projection methods includes spherical projection, bird-eye view projection, etc. Although this process makes the point cloud suitable for the 2D CNN-based networks, it inevitably alters and abandons the 3D topology and geometric relations. A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space. In this work, we first perform an in-depth analysis for different representations and backbones in 2D and 3D spaces, and reveal the effectiveness of 3D representations and networks on LiDAR segmentation. Then, we develop a 3D cylinder partition and a 3D cylinder convolution based framework, termed as Cylinder3D, which exploits the 3D topology relations and structures of driving-scene point clouds. Moreover, a dimension-decomposition based context modeling module is introduced to explore the high-rank context information in point clouds in a progressive manner. We evaluate the proposed model on a large-scale driving-scene dataset, i.e. SematicKITTI. Our method achieves state-of-the-art performance and outperforms existing methods by 6% in terms of mIoU.

Motivation & Objective

• Assess the limitations of 2D projections for driving-scene LiDAR segmentation and quantify the benefits of 3D processing.
• Investigate point representations and backbones in 2D vs 3D spaces to identify effective configurations.
• Develop a cylinder-based voxelization and a 3D CNN backbone tailored to driving-scene point clouds.
• Introduce an asymmetric residual block and a dimension-decomposition based context modeling module to improve efficiency and context modeling.

Proposed method

• Introduce a 3D cylinder partition to voxelize LiDAR points in cylindrical coordinates, producing a 3D representation that preserves topology.
• Process the cylinder-based representation with a 3D U-Net backbone using sparse 3D convolutions.
• Replace standard residual blocks with an asymmetric residual block to better match cuboid driving objects and reduce computation.
• Attach a dimension-decomposition based context modeling (DDCM) module that decomposes high-rank context into three low-rank components (3x1x1, 1x3x1, 1x1x3) and fuses them.
• Train with a weighted cross-entropy loss plus Lovász-softmax loss to optimize per-point accuracy and mIoU.
• Use Adam optimizer with an initial learning rate of 0.001.

Experimental results

Research questions

• RQ1 Does processing LiDAR data in 3D (via cylinder partition) outperform 2D projection-based representations for driving-scene segmentation?
• RQ2 How does cylinder-based voxelization compare to Cartesian voxelization in capturing 3D topology of outdoor scenes?
• RQ3 What are the contributions of the asymmetric residual block and the dimension-decomposition context modeling to overall performance?
• RQ4 What performance gains does Cylinder3D achieve on SemanticKITTI compared to prior methods?

Key findings

• Cylinder3D achieves state-of-the-art performance on SemanticKITTI, outperforming existing methods by a notable margin in mIoU.
• 3D cylinder partition plus 3D convolutions significantly improve results over 2D backbones applied to projection-based representations.
• Replacing standard residual blocks with asymmetric residual blocks yields about 1.5% mIoU gain.
• Incorporating the dimension-decomposition context modeling module further boosts mIoU, with ablations showing substantial gains.
• Flip test provides an additional small mIoU improvement when ensembling multiple augmented predictions.