[Paper Review] PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval

Wenxiao Zhang, Chunxia Xiao|arXiv (Cornell University)|Apr 22, 2019

3D Shape Modeling and AnalysisEngineering40 references22 citations

TL;DR

This paper proposes PCAN, a 3D attention network that learns context-aware attention maps to improve point cloud-based place recognition by reweighting local features based on contextual information. By integrating attention into NetVLAD for global descriptor learning, PCAN achieves state-of-the-art performance on benchmark datasets without additional supervision.

ABSTRACT

Point cloud based retrieval for place recognition is an emerging problem in vision field. The main challenge is how to find an efficient way to encode the local features into a discriminative global descriptor. In this paper, we propose a Point Contextual Attention Network (PCAN), which can predict the significance of each local point feature based on point context. Our network makes it possible to pay more attention to the task-relevent features when aggregating local features. Experiments on various benchmark datasets show that the proposed network can provide outperformance than current state-of-the-art approaches.

Motivation & Objective

• To address the challenge of encoding local point features into a discriminative global descriptor for 3D point cloud-based place recognition.
• To improve feature aggregation by learning which local features are most relevant to the task using contextual information.
• To reduce sensitivity to noisy or time-varying elements (e.g., pedestrians, floating points) in point clouds during retrieval.
• To develop a self-supervised attention mechanism that does not require additional annotations beyond localization tags.
• To enhance robustness and discriminative power in complex and variable environments, especially under changing lighting and occlusion conditions.

Proposed method

• PCAN uses a PointNet-based backbone to extract per-point local features from raw 3D point clouds.
• A context-aware attention module computes an attention map by aggregating multi-scale contextual features using ball query-based neighborhood pooling.
• The attention map assigns per-point weights based on contextual relevance, emphasizing stable, discriminative structures like building corners.
• The weighted features are then aggregated via a modified NetVLAD layer to produce a compact, discriminative global descriptor.
• The network is trained end-to-end using only localization tags (no additional supervision), with L2 normalization applied to the final descriptor.
• Multi-resolution grouping (MRG) and multi-scale grouping (MSG) are evaluated as context aggregation strategies to balance stability and discriminative power.

Experimental results

Research questions

• RQ1 Can context-aware attention learning improve the discriminative quality of global descriptors in 3D point cloud retrieval?
• RQ2 How does incorporating contextual information from neighboring points affect attention map quality and retrieval performance?
• RQ3 To what extent can a self-supervised attention mechanism reduce sensitivity to noisy or dynamic elements in point clouds?
• RQ4 How does the proposed PCAN compare to attention mechanisms that ignore contextual information (e.g., PAN) in terms of accuracy and robustness?
• RQ5 What is the impact of different grouping strategies (e.g., MRG, MSG) on the stability and performance of the attention mechanism?

Key findings

• PCAN achieves 83.81% average recall at top 1% on the Oxford dataset, outperforming the baseline PointNetVLAD and the ablation model PAN.
• On the Oxford dataset, PCAN improves top-1 recall by 3.6 percentage points over PAN, demonstrating the value of context-aware attention.
• PCAN maintains strong performance on in-house datasets (e.g., 69.05% recall at top 1% on U.S. dataset), even with less training data and more complex scenes.
• The attention maps successfully suppress noisy points and dynamic objects (e.g., pedestrians, floating debris), as visualized in the results.
• The ablation study confirms that context aggregation significantly improves performance, with PCAN outperforming PAN across all datasets.
• PCAN shows a favorable trade-off between stability (ignoring unstable features like leaves) and discriminative ability (highlighting unique structures like corners).