[Paper Review] A Closer Look at Local Aggregation Operators in Point Cloud Analysis

Ze Liu, Hu Han+3 more|arXiv (Cornell University)|Jul 2, 2020

3D Shape Modeling and Analysis36 references20 citations

TL;DR

This paper proposes Position Pooling (PosPool), a lightweight local aggregation operator for point cloud processing that combines point features and 3D relative coordinates via element-wise multiplication followed by average pooling—achieving state-of-the-art performance on ModelNet40, S3DIS, and PartNet benchmarks, outperforming complex, learnable operators, especially by 7.4 mIoU on PartNet.

ABSTRACT

Recent advances of network architecture for point cloud processing are mainly driven by new designs of local aggregation operators. However, the impact of these operators to network performance is not carefully investigated due to different overall network architecture and implementation details in each solution. Meanwhile, most of operators are only applied in shallow architectures. In this paper, we revisit the representative local aggregation operators and study their performance using the same deep residual architecture. Our investigation reveals that despite the different designs of these operators, all of these operators make surprisingly similar contributions to the network performance under the same network input and feature numbers and result in the state-of-the-art accuracy on standard benchmarks. This finding stimulate us to rethink the necessity of sophisticated design of local aggregation operator for point cloud processing. To this end, we propose a simple local aggregation operator without learnable weights, named Position Pooling (PosPool), which performs similarly or slightly better than existing sophisticated operators. In particular, a simple deep residual network with PosPool layers achieves outstanding performance on all benchmarks, which outperforms the previous state-of-the methods on the challenging PartNet datasets by a large margin (7.4 mIoU). The code is publicly available at https://github.com/zeliu98/CloserLook3D

Motivation & Objective

• To fairly evaluate the impact of different local aggregation operators on point cloud network performance by eliminating architectural and implementation variability.
• To investigate whether sophisticated, learnable aggregation operators are truly necessary for high performance in deep networks.
• To explore whether a minimal, non-parametric operator can match or surpass existing complex designs.
• To establish a common benchmark for fair comparison of local aggregation operators under consistent deep residual architectures.

Proposed method

• Proposes Position Pooling (PosPool), a weight-free local aggregation operator that performs element-wise multiplication between neighbor point features and their 3D relative coordinates.
• Applies average pooling over the resulting products to aggregate neighborhood information into a single feature vector for the center point.
• Employs a deep residual network with PosPool layers as the backbone, using shared weights across all layers and no learnable parameters in the aggregation module.
• Uses a unified experimental setup across three benchmarks—ModelNet40, S3DIS, and PartNet—ensuring consistent data splitting, input resolution, and training/inference protocols.
• Visualizes activation maps before final prediction to compare feature learning patterns across different operators.
• Evaluates performance across varying network depth, width, and bottleneck ratios to assess robustness and adaptability.

Experimental results

Research questions

• RQ1 Do different local aggregation operators contribute differently to network performance when evaluated under identical deep residual architectures?
• RQ2 Can a simple, non-parametric operator like PosPool achieve performance comparable to or better than complex, learnable operators?
• RQ3 How does the performance of existing operators vary with changes in model capacity (depth, width, bottleneck ratio)?
• RQ4 Do different operators learn similar or distinct representations from the same input point cloud?

Key findings

• All evaluated local aggregation operators achieve state-of-the-art performance on ModelNet40, S3DIS, and PartNet when appropriately tuned, despite differing designs and motivations.
• Position Pooling (PosPool) achieves the best performance on the challenging PartNet dataset, outperforming previous state-of-the-art methods by 7.4 mIoU on the validation set.
• PosPool shows superior stability across varying model capacities, while more complex operators like 'AdaptWeight' suffer significant accuracy drops when model depth or width is reduced.
• Visualization of activation maps reveals that different operators learn similar high-energy regions in the input point cloud, suggesting convergence in feature representation despite architectural differences.
• The absence of learnable weights in PosPool leads to minimal memory and computational cost, with time complexity O(ndK) and space complexity O(0).