[Paper Review] Decoupled Local Aggregation for Point Cloud Learning

Binjie Chen, Yunzhou Xia+3 more|arXiv (Cornell University)|Aug 31, 2023

3D Surveying and Cultural Heritage12 citations

TL;DR

The paper introduces DeLA, a lightweight decoupled local aggregation framework for point clouds that encodes spatial information in features and uses simple pooling with edge max-pooling, achieving state-of-the-art results with reduced latency.

ABSTRACT

The unstructured nature of point clouds demands that local aggregation be adaptive to different local structures. Previous methods meet this by explicitly embedding spatial relations into each aggregation process. Although this coupled approach has been shown effective in generating clear semantics, aggregation can be greatly slowed down due to repeated relation learning and redundant computation to mix directional and point features. In this work, we propose to decouple the explicit modelling of spatial relations from local aggregation. We theoretically prove that basic neighbor pooling operations can too function without loss of clarity in feature fusion, so long as essential spatial information has been encoded in point features. As an instantiation of decoupled local aggregation, we present DeLA, a lightweight point network, where in each learning stage relative spatial encodings are first formed, and only pointwise convolutions plus edge max-pooling are used for local aggregation then. Further, a regularization term is employed to reduce potential ambiguity through the prediction of relative coordinates. Conceptually simple though, experimental results on five classic benchmarks demonstrate that DeLA achieves state-of-the-art performance with reduced or comparable latency. Specifically, DeLA achieves over 90\% overall accuracy on ScanObjectNN and 74\% mIoU on S3DIS Area 5. Our code is available at https://github.com/Matrix-ASC/DeLA .

Motivation & Objective

• Motivate reducing complexity in local aggregation by decoupling spatial relation learning from aggregation.
• Propose spatial encoding to preserve local spatial information without learning relations at every step.
• Build DeLA as a lightweight network using interleaved pointwise operations and edge pooling.
• Regularize to encourage utilization of spatial encoding and reduce ambiguity.
• Demonstrate strong performance on standard point cloud benchmarks across classification and segmentation tasks.

Proposed method

• Define spatial encoding to recover relative coordinates locally from encoded point features.
• Decompose standard local aggregation into a set-embedding based message passing with g, M, h (Equation 7).
• Instantiate DeLA with Local Feature Propagation (LFP) and edge max-pooling for aggregation.
• Form spatial encoding via a small PointNet that processes neighbor relations at each stage.
• Downsample with FPS or grid subsampling and use skip connections to preserve source features.
• Apply a regularization term that predicts relative coordinates between neighboring points and decays this term over training.

Experimental results

Research questions

• RQ1 Can decoupling explicit spatial relation learning from local aggregation maintain or improve feature fusion clarity in point clouds?
• RQ2 Does embedding spatial information into point features allow basic pooling to achieve competitive or superior performance?
• RQ3 What is the impact of different spatial encodings and aggregation operators on segmentation and classification tasks?
• RQ4 How does DeLA compare to state-of-the-art methods in terms of accuracy and efficiency across common benchmarks?

Key findings

• DeLA achieves state-of-the-art performance with reduced or comparable latency across benchmarks.
• On S3DIS Area 5, DeLA attains OA 92.2, mAcc 80.0, mIoU 74.1, with throughput 468 instances/sec.
• On ScanNet v2, DeLA achieves the best validation mIoU among reported methods with a compact parameter count (8.0M).
• On ShapeNetPart, DeLA reaches ins. mIoU 87.0 and cat. mIoU 85.8 without voting, closely rivaling or surpassing many methods.
• On ScanObjectNN and ModelNet40, DeLA achieves OA 90.4 and 94.0 respectively with competitive mAcc 89.3 and 92.2.
• The ablations show relative spatial encoding (especially rel-first and rel-rest) is effective, and edge max-pooling provides robust aggregation.