[Paper Review] APPT : Asymmetric Parallel Point Transformer for 3D Point Cloud Understanding

Hengjia Li, Zhengkai Tu+6 more|arXiv (Cornell University)|Mar 31, 2023

3D Shape Modeling and Analysis4 citations

TL;DR

This paper proposes Asymmetric Parallel Point Transformer (APPT), a novel 3D point cloud network that enhances global context modeling via Global Pivot Attention (GPA) and asymmetric parallel branches to integrate local and global features. By using farthest-point-sampled pivots for efficient long-range attention and a ramped channel/sampling ratio strategy, APPT achieves state-of-the-art performance on S3DIS, ModelNet40, and ShapeNet benchmarks with mIoU improvements up to 72.3%.

ABSTRACT

Transformer-based networks have achieved impressive performance in 3D point cloud understanding. However, most of them concentrate on aggregating local features, but neglect to directly model global dependencies, which results in a limited effective receptive field. Besides, how to effectively incorporate local and global components also remains challenging. To tackle these problems, we propose Asymmetric Parallel Point Transformer (APPT). Specifically, we introduce Global Pivot Attention to extract global features and enlarge the effective receptive field. Moreover, we design the Asymmetric Parallel structure to effectively integrate local and global information. Combined with these designs, APPT is able to capture features globally throughout the entire network while focusing on local-detailed features. Extensive experiments show that our method outperforms the priors and achieves state-of-the-art on several benchmarks for 3D point cloud understanding, such as 3D semantic segmentation on S3DIS, 3D shape classification on ModelNet40, and 3D part segmentation on ShapeNet.

Motivation & Objective

• To address the limited effective receptive field in existing Transformer-based 3D point cloud networks, which often neglect global dependencies.
• To improve the integration of local geometric details and global semantic context in 3D point cloud representation learning.
• To design a computationally efficient global attention mechanism that scales well to large point clouds.
• To validate the effectiveness of asymmetric parallel architecture with progressive channel and sampling ratios across network depth.

Proposed method

• Proposes Global Pivot Attention (GPA), a scalar-attention mechanism using farthest-point-sampled pivot points to model long-range dependencies efficiently.
• Introduces an asymmetric parallel block that processes local features via Local Group Attention (LGA) and global features via GPA in separate, parallel streams.
• Employs a ramped channel-ratio strategy that gradually increases the proportion of channels allocated to the global branch from bottom to top layers.
• Applies a ramped sampling-ratio strategy that increases the number of pivot points in the global branch as the network depth increases.
• Uses simple concatenation of local and global feature outputs as the fusion strategy, outperforming element-wise sum or SE-based concatenation.
• Designs a full network framework, APPT, that integrates these components for end-to-end 3D point cloud understanding.

Experimental results

Research questions

• RQ1 Can a global attention mechanism based on sampled pivots effectively enlarge the effective receptive field in 3D point cloud networks?
• RQ2 How can local and global features be effectively combined in a deep 3D point cloud network without significant computational overhead?
• RQ3 Does a progressive increase in global feature capacity from shallow to deep layers improve performance on 3D understanding tasks?
• RQ4 How does the proposed asymmetric parallel design compare to serial or symmetric designs in terms of feature integration and accuracy?

Key findings

• APPT achieves 72.3% mIoU on the S3DIS 3D semantic segmentation benchmark, outperforming prior methods including Point Transformer and Stratified Transformer.
• The ablation study confirms that the parallel design with GPA yields a 72.3% mIoU, significantly outperforming the serial design (70.9% mIoU).
• GPA outperforms other global modules such as Stratified Transformer (ST), Squeeze-Extraction (SE), and Global Context (GC), especially in long-range dependency modeling.
• The ramped channel-ratio strategy ($C_g/C\uparrow$) achieves the highest mIoU (72.3%) and is more effective than fixed or decreasing ratios.
• The ramped sampling-ratio strategy ($M/N\uparrow$) yields the best performance (72.3% mIoU), demonstrating the benefit of increasing pivot points in deeper layers.
• Simple concatenation of local and global features achieves the best performance (72.3% mIoU), outperforming fusion with Squeeze-Extraction or element-wise summation.