[Paper Review] Deep Positional and Relational Feature Learning for Rotation-Invariant Point Cloud Analysis

Ruixuan Yu, Xin Wei+2 more|arXiv (Cornell University)|Nov 18, 2020

3D Shape Modeling and Analysis41 references4 citations

TL;DR

This paper proposes PR-invNet, a rotation-invariant deep network for 3D point cloud analysis that jointly learns positional and relational features through two novel modules: a Positional Feature Embedding (PFE) block and a Relational Feature Embedding (RFE) block. The PFE-block uses a pose expander with the alternating group $A_5$ to generate a rotation-invariant pose space and a multi-head score network to select the most discriminative shape pose, while the RFE-block enhances features using relation-aware convolutions. The method achieves state-of-the-art performance on ModelNet40 and ShapeNetPart, with 89.2% accuracy on ModelNet40 and 89.5% on PartNet, demonstrating robustness to rotation and noise.

ABSTRACT

In this paper we propose a rotation-invariant deep network for point clouds analysis. Point-based deep networks are commonly designed to recognize roughly aligned 3D shapes based on point coordinates, but suffer from performance drops with shape rotations. Some geometric features, e.g., distances and angles of points as inputs of network, are rotation-invariant but lose positional information of points. In this work, we propose a novel deep network for point clouds by incorporating positional information of points as inputs while yielding rotation-invariance. The network is hierarchical and relies on two modules: a positional feature embedding block and a relational feature embedding block. Both modules and the whole network are proven to be rotation-invariant when processing point clouds as input. Experiments show state-of-the-art classification and segmentation performances on benchmark datasets, and ablation studies demonstrate effectiveness of the network design.

Motivation & Objective

• To address the rotation-sensitivity of existing point-based deep networks that rely on raw coordinates and fail under arbitrary 3D rotations.
• To preserve discriminative positional information while achieving rigorous rotation-invariance in deep feature learning for point clouds.
• To design a hierarchical network architecture that integrates both positional and relational features for improved 3D shape recognition.
• To overcome limitations of geometric feature-only approaches that lose spatial context and fail to distinguish similar local structures.
• To develop a scalable and efficient rotation-invariant network that avoids costly data augmentation and generalizes well under noise and unseen rotations.

Proposed method

• The PFE-block constructs a rotation-invariant pose space using the alternating group $A_5$ of a dodecahedron, mapping the input point cloud to 120 rotated poses.
• A multi-head score network ($\Psi$) acts as a pose selector, choosing the most representative pose from the rot-space based on maximum score across multiple heads.
• The selected pose's coordinates are used to extract positional features via a positional feature extractor, ensuring discriminative representation of local geometry.
• The RFE-block applies relation-aware convolutions using three components: relative point positions, geometric features (e.g., distances, angles), and point features, with learnable weights.
• All network components, including PFE and RFE blocks, are mathematically proven to be rotation-invariant under the SO(3) group.
• The full network, PR-invNet, is trained end-to-end for classification and segmentation tasks using standard benchmarks.

Experimental results

Research questions

• RQ1 Can a deep network learn rotation-invariant features from raw point coordinates without relying on data augmentation?
• RQ2 How can positional information be preserved while achieving invariance to arbitrary 3D rotations in point cloud analysis?
• RQ3 What is the impact of using a multi-head score network for pose selection in a rotation-invariant feature learning framework?
• RQ4 How do relational features based on point positions and geometric relations improve performance compared to geometric features alone?
• RQ5 To what extent does the proposed architecture maintain robustness under noise and arbitrary rotations?

Key findings

• PR-invNet achieves 89.2% classification accuracy on ModelNet40, outperforming prior state-of-the-art methods under the z/SO(3) rotation mode.
• The ablation study confirms that the full PFE-block with pose selection outperforms feature aggregation via max or average pooling, achieving 88.5% accuracy with 58% less GPU memory and 49% less time.
• Using a multi-head score network ($\Psi$) with 200 heads yields higher accuracy than single-head or other head configurations, demonstrating improved pose selection capability.
• The RFE-block with all three components (position, geometry, feature) achieves the best performance, while removing any component leads to a drop in accuracy.
• The network maintains robustness under noise, achieving 88.6% accuracy with Gaussian noise (std = 0.01), indicating strong generalization to perturbed inputs.
• Visualization shows that each head in the multi-head score network selects poses with similar orientations, suggesting that the network learns to cluster shapes into meaningful pose groups.