[Paper Review] Local Spectral Graph Convolution for Point Set Feature Learning

Chu Wang, Babak Samari+1 more|arXiv (Cornell University)|Mar 15, 2018

3D Shape Modeling and Analysis18 references61 citations

TL;DR

The paper introduces local spectral graph convolution on k-NN neighborhoods with recursive cluster pooling to learn joint neighborhood features for point clouds, achieving state-of-the-art results in classification and segmentation compared with PointNet++.

ABSTRACT

Feature learning on point clouds has shown great promise, with the introduction of effective and generalizable deep learning frameworks such as pointnet++. Thus far, however, point features have been abstracted in an independent and isolated manner, ignoring the relative layout of neighboring points as well as their features. In the present article, we propose to overcome this limitation by using spectral graph convolution on a local graph, combined with a novel graph pooling strategy. In our approach, graph convolution is carried out on a nearest neighbor graph constructed from a point's neighborhood, such that features are jointly learned. We replace the standard max pooling step with a recursive clustering and pooling strategy, devised to aggregate information from within clusters of nodes that are close to one another in their spectral coordinates, leading to richer overall feature descriptors. Through extensive experiments on diverse datasets, we show a consistent demonstrable advantage for the tasks of both point set classification and segmentation.

Motivation & Objective

• Motivate robust feature learning on unorganized 3D point clouds by incorporating neighborhood structure.
• Learn joint features from k-NN neighborhoods instead of isolated point-wise features.
• Replace max pooling with recursive cluster pooling to capture multiple salient local features.
• Develop an end-to-end trainable framework that builds graphs on-the-fly without offline precomputation.

Proposed method

• Build a local k-NN graph for each point's neighborhood and compute its Laplacian.
• Apply graph Fourier transform to obtain spectral representation of neighborhood features.
• Learn a spectral filter in the graph Fourier domain and transform back to spatial domain.
• Replace max pooling with recursive spectral cluster pooling guided by the Fiedler vector to aggregate within clusters of similar spectral coordinates.
• Integrate the local spectral convolution into a PointNet++-like framework with end-to-end training and no offline graph precomputation.

Experimental results

Research questions

• RQ1 Does local spectral graph convolution improve feature learning over traditional point-wise MLPs in PointNet++?
• RQ2 Can recursive cluster pooling of spectral coordinates better capture multiple salient local features than standard max pooling?
• RQ3 Is the approach scalable with respect to input size and compatible with end-to-end training without offline graph precomputation?
• RQ4 What performance gains are observed on common 3D point cloud benchmarks for classification and segmentation?

Key findings

• The spectral graph convolution on local k-NN graphs outperforms point-wise MLPs (e.g., 4l-spec-max vs 4l-pointnet++ on ModelNet40).
• Recursive cluster pooling provides additional gains over max pooling, improving both classification and segmentation results.
• The proposed 4l-spec-cp model achieves competitive or state-of-the-art results across ModelNet40, ShapeNet, and ScanNet datasets.
• The method yields improvements on 3D shape benchmarks (McGill Shape Benchmark, ModelNet40) and large-scale indoor segmentation (ScanNet) compared to prior approaches.
• Training time remains comparable to PointNet++ despite eigen-decomposition in local graphs, due to small k in local neighborhoods.