[Paper Review] APP-Net: Auxiliary-point-based Push and Pull Operations for Efficient Point Cloud Classification

Tao Lu, Chunxu Liu+3 more|arXiv (Cornell University)|May 2, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

APP-Net proposes a linear-complexity local aggregator for 3D point cloud classification using auxiliary points to decouple feature aggregation from receptive field expansion, eliminating redundant computation and memory usage. It achieves over 10,000 samples per second inference speed with under 10GB GPU memory, outperforming prior methods in efficiency while maintaining state-of-the-art accuracy on ModelNet40.

ABSTRACT

Aggregating neighbor features is essential for point cloud classification. In the existing work, each point in the cloud may inevitably be selected as the neighbors of multiple aggregation centers, as all centers will gather neighbor features from the whole point cloud independently. Thus each point has to participate in the calculation repeatedly and generates redundant duplicates in the memory, leading to intensive computation costs and memory consumption. Meanwhile, to pursue higher accuracy, previous methods often rely on a complex local aggregator to extract fine geometric representation, which further slows down the classification pipeline. To address these issues, we propose a new local aggregator of linear complexity for point cloud classification, coined as APP. Specifically, we introduce an auxiliary container as an anchor to exchange features between the source point and the aggregating center. Each source point pushes its feature to only one auxiliary container, and each center point pulls features from only one auxiliary container. This avoids the re-computation issue of each source point. To facilitate the learning of the local structure of cloud point, we use an online normal estimation module to provide the explainable geometric information to enhance our APP modeling capability. Our built network is more efficient than all the previous baselines with a clear margin while still consuming a lower memory. Experiments on both synthetic and real datasets demonstrate that APP-Net reaches comparable accuracies to other networks. It can process more than 10,000 samples per second with less than 10GB of memory on a single GPU. We will release the code in https://github.com/MCG-NJU/APP-Net.

Motivation & Objective

• To address the high computational and memory cost caused by redundant neighbor feature aggregation in existing point cloud classification networks.
• To decouple feature aggregation from receptive field expansion to reduce repeated computation and memory footprint.
• To design a lightweight, efficient local aggregator that maintains high accuracy while significantly improving inference speed and memory efficiency.
• To enable real-time inference on large-scale point clouds by minimizing redundant operations and leveraging structured feature exchange via auxiliary containers.

Proposed method

• Introduce an auxiliary point container as a shared intermediary to mediate feature exchange between source points and aggregation centers.
• Each source point pushes its feature to only one auxiliary container, and each center pulls features from only one such container, eliminating multi-center neighbor conflicts.
• Use a position-aware, online normal estimation module to provide geometric context that enhances local feature modeling.
• Apply a learnable delta function to update features after pull operations, using concatenation for improved representation learning.
• Adopt adaptive down-sampling and rate control (rd and ra) to manage receptive field size and maintain performance across network depth.
• Combine average and max pooling in the final aggregation step to preserve both contextual and discriminative features.

Experimental results

Research questions

• RQ1 Can a linear-complexity local aggregator be designed to reduce redundant computation and memory usage in point cloud classification?
• RQ2 Does decoupling feature aggregation from receptive field expansion improve efficiency without sacrificing accuracy?
• RQ3 Can auxiliary points serve as an effective, lightweight mechanism for structured feature exchange in irregular point clouds?
• RQ4 How does the integration of geometric priors (via normal estimation) affect the performance of an efficient local aggregator?
• RQ5 To what extent can inference speed and memory consumption be reduced while maintaining competitive classification accuracy?

Key findings

• APP-Net processes over 10,000 point cloud samples per second on a single GPU, significantly outperforming prior methods in inference speed.
• The model consumes less than 10GB of GPU memory, demonstrating superior memory efficiency compared to existing baselines.
• On the ModelNet40 dataset, APP-Net achieves an overall accuracy (OA) of 84.3%, matching state-of-the-art performance while using a simpler architecture.
• The ablation study confirms that concatenation with the delta function after the pull step yields the best performance, outperforming residual or identity connections.
• The 3-layer network with rd = [8,8,8] and ra = [64,64,64] achieves the highest accuracy, indicating optimal depth and receptive field configuration.
• Combining average and max pooling yields the best performance (84.3% OA), surpassing individual pooling strategies or position-adaptive pooling.