[Paper Review] Relation-Shape Convolutional Neural Network for Point Cloud Analysis

Yongcheng Liu, Bin Fan+2 more|arXiv (Cornell University)|Apr 16, 2019

3D Shape Modeling and AnalysisEngineering54 references91 citations

TL;DR

RS-CNN extends regular grid CNN to irregular point clouds by learning from geometric relations among points, achieving state-of-the-art results on classification, segmentation, and normal estimation tasks.

ABSTRACT

Point cloud analysis is very challenging, as the shape implied in irregular points is difficult to capture. In this paper, we propose RS-CNN, namely, Relation-Shape Convolutional Neural Network, which extends regular grid CNN to irregular configuration for point cloud analysis. The key to RS-CNN is learning from relation, i.e., the geometric topology constraint among points. Specifically, the convolutional weight for local point set is forced to learn a high-level relation expression from predefined geometric priors, between a sampled point from this point set and the others. In this way, an inductive local representation with explicit reasoning about the spatial layout of points can be obtained, which leads to much shape awareness and robustness. With this convolution as a basic operator, RS-CNN, a hierarchical architecture can be developed to achieve contextual shape-aware learning for point cloud analysis. Extensive experiments on challenging benchmarks across three tasks verify RS-CNN achieves the state of the arts.

Motivation & Objective

• Address permutation invariance, transformation robustness, and shape-aware representation for irregular point clouds.
• Propose a relation-shape convolution (RS-Conv) that learns high-level relations between a centroid point and its neighbors.
• Build a hierarchical RS-CNN architecture to enable contextual shape-aware learning for point clouds.
• Demonstrate state-of-the-art performance across classification, segmentation, and normal estimation tasks.

Proposed method

• Construct local spherical neighborhoods around a sampled centroid x_i with neighbors N(x_i).
• Replace traditional per-neighbor weights with a relation-mapping w_ij = M(h_ij) where h_ij encodes low-level geometric priors.
• Aggregate transformed neighbor features with a symmetric function A (e.g., max) to obtain f_{P_sub}.
• Apply a channel-raising MLP to f_{P_sub} for higher-level representations.
• Use three-scale neighborhoods with a shared weight to capture multi-scale geometry.
• Train end-to-end for tasks like classification and segmentation using xyz coordinates as input.

Experimental results

Research questions

• RQ1 How can grid CNN concepts be extended to irregular point clouds via learning from geometric relations?
• RQ2 Can relation-shape convolution provide permutation invariance and robustness to rigid transformations while yielding shape-aware representations?
• RQ3 What is RS-CNN’s performance relative to state-of-the-art methods on classification, segmentation, and normal estimation tasks?

Key findings

• RS-CNN achieves 93.6% accuracy on ModelNet40 with xyz input (1k points) after 10 voting tests.
• RS-CNN reduces the error of PointNet++ by 31.2% on ModelNet40 classification.
• On ShapeNet part segmentation, RS-CNN attains class mIoU of 84.0% and instance mIoU of 86.2%.
• For normal estimation on ModelNet40 using RS-CNN, the reported error is 0.15, significantly better than PointNet++ (0.29).
• RS-CNN demonstrates robustness to sampling density and geometric perturbations, maintaining competitive performance when input points are reduced.