[Paper Review] Discrete Rotation Equivariance for Point Cloud Recognition

Jiaxin Li, Yingcai Bi+1 more|arXiv (Cornell University)|Mar 31, 2019

3D Shape Modeling and Analysis43 references4 citations

TL;DR

This paper proposes a discrete rotation equivariance method for point cloud recognition by applying predefined rotations to input point clouds and leveraging feature permutation to achieve SO(2)/SO(3) equivariance. The approach significantly improves classification accuracy on rotated inputs—especially under SO(3) rotations—by up to 10.8% on ModelNet40 when integrated with multi-hierarchical networks like PointNet-FE, while revealing that vanilla PointNet fails due to its global-pooling design.

ABSTRACT

Despite the recent active research on processing point clouds with deep networks, few attention has been on the sensitivity of the networks to rotations. In this paper, we propose a deep learning architecture that achieves discrete $\mathbf{SO}(2)$/$\mathbf{SO}(3)$ rotation equivariance for point cloud recognition. Specifically, the rotation of an input point cloud with elements of a rotation group is similar to shuffling the feature vectors generated by our approach. The equivariance is easily reduced to invariance by eliminating the permutation with operations such as maximum or average. Our method can be directly applied to any existing point cloud based networks, resulting in significant improvements in their performance for rotated inputs. We show state-of-the-art results in the classification tasks with various datasets under both $\mathbf{SO}(2)$ and $\mathbf{SO}(3)$ rotations. In addition, we further analyze the necessary conditions of applying our approach to PointNet based networks. Source codes at https://github.com/lijx10/rot-equ-net

Motivation & Objective

• To address the lack of rotation equivariance in deep learning models for point cloud recognition.
• To develop a generalizable method that achieves discrete SO(2)/SO(3) rotation equivariance without architectural overhaul.
• To improve performance on rotated point cloud classification tasks, especially under SO(3) rotations.
• To analyze the structural requirements for integrating the method into existing networks like PointNet.
• To provide a systematic, explainable alternative to data augmentation for handling rotational variations.

Proposed method

• Define a finite k-element rotation group R = {r_i, i ∈ [0, ..., k-1]} for discrete rotations in SO(2) or SO(3).
• Apply each rotation r_i to the input point cloud and feed all rotated versions into the same network to generate k feature vectors.
• Prove that rotating the input point cloud with elements of R results in a permutation of the k output feature vectors, establishing rotation equivariance.
• Achieve invariance by applying permutation-equivariant operations like max or average pooling over the k features.
• Integrate the method into existing point cloud networks (e.g., PointNet, SO-Net, PointNet-FE) without modifying their core architecture.
• Use polar coordinate transformation as a future direction to convert rotation equivariance into translation equivariance.

Experimental results

Research questions

• RQ1 Can discrete SO(2)/SO(3) rotation equivariance be achieved in point cloud networks using a general, plug-in method?
• RQ2 How does the proposed method improve performance on rotated point cloud classification compared to data augmentation?
• RQ3 Why does the method improve performance on PointNet-FE and SO-Net but degrade it on vanilla PointNet?
• RQ4 What architectural properties are necessary for a network to benefit from the proposed equivariance design?
• RQ5 Can the method be extended to continuous rotation equivariance using coordinate transformations?

Key findings

• The proposed method achieves state-of-the-art performance on ModelNet40 and MNIST under both SO(2) and SO(3) rotations.
• On ModelNet40, the method improves classification accuracy by 10.8% under SO(3) rotation when applied to PointNet-FE.
• For SO-Net, the method improves accuracy by 4.4% under SO(3) rotation, demonstrating consistent gains on multi-hierarchical networks.
• Vanilla PointNet shows a 19.9% performance drop under SO(3) rotation after applying the method, due to its global-pooling design limiting feature interaction.
• The improvement is more pronounced under SO(3) than SO(2), likely because data augmentation is already effective in 2D rotation scenarios.
• Increasing the rotation group size improves performance, but with diminishing returns, and computational cost scales linearly with group size.