[Paper Review] Multi-Resolution Graph Neural Network for Large-Scale Pointcloud Segmentation

Liuyue Xie, Tomotake Furuhata+1 more|arXiv (Cornell University)|Sep 18, 2020

3D Shape Modeling and Analysis28 references4 citations

TL;DR

This paper proposes MuGNet, a memory-efficient, end-to-end multi-resolution graph neural network for large-scale point cloud semantic segmentation. By clustering dense point clouds into geometrically similar groups and using a bidirectional graph convolutional network to fuse features across resolutions, MuGNet processes up to 45 room scans simultaneously on a single 11GB GPU, achieving 88.5% overall accuracy and 69.8% mIOU on S3DIS—surpassing prior graph-based methods by 3% and 7.7% respectively.

ABSTRACT

In this paper, we propose a multi-resolution deep-learning architecture to semantically segment dense large-scale pointclouds. Dense pointcloud data require a computationally expensive feature encoding process before semantic segmentation. Previous work has used different approaches to drastically downsample from the original pointcloud so common computing hardware can be utilized. While these approaches can relieve the computation burden to some extent, they are still limited in their processing capability for multiple scans. We present MuGNet, a memory-efficient, end-to-end graph neural network framework to perform semantic segmentation on large-scale pointclouds. We reduce the computation demand by utilizing a graph neural network on the preformed pointcloud graphs and retain the precision of the segmentation with a bidirectional network that fuses feature embedding at different resolutions. Our framework has been validated on benchmark datasets including Stanford Large-Scale 3D Indoor Spaces Dataset(S3DIS) and Virtual KITTI Dataset. We demonstrate that our framework can process up to 45 room scans at once on a single 11 GB GPU while still surpassing other graph-based solutions for segmentation on S3DIS with an 88.5\% (+3\%) overall accuracy and 69.8\% (+7.7\%) mIOU accuracy.

Motivation & Objective

• Address the high memory and computational demands of segmenting dense, large-scale point clouds with billions of points.
• Overcome the limitations of drastic downsampling and sliding window methods that degrade segmentation accuracy by losing fine geometric details.
• Develop a permutation-invariant framework that avoids artificial structuring of point clouds into voxels or meshes.
• Enable end-to-end, real-time processing of multiple large-scale scans on standard GPU hardware.
• Preserve fine-grained geometric details while reducing memory usage through multi-resolution clustering and feature fusion.

Proposed method

• Transform dense point clouds into semantically similar clusters to drastically reduce GPU memory usage.
• Construct point cloud graphs from clustered points and apply graph convolutional networks (GCNs) that are invariant to point ordering.
• Implement a bidirectional multi-resolution fusion network that combines features from different resolution levels using forward and backward paths.
• Use residual connections in the backbone to stabilize training and improve feature representation across multiple GCN layers.
• Apply a multi-scale clustering strategy to generate hierarchical point cluster representations for improved geometric context modeling.
• Train the end-to-end network using cross-entropy loss with a 6-fold cross-validation setup on S3DIS and Virtual KITTI datasets.

Experimental results

Research questions

• RQ1 Can a multi-resolution graph neural network achieve high-accuracy semantic segmentation on dense, large-scale point clouds while minimizing GPU memory usage?
• RQ2 How does bidirectional feature fusion across multiple graph resolutions improve segmentation performance compared to single-resolution or unidirectional fusion?
• RQ3 To what extent can the proposed framework process multiple large-scale scans simultaneously without sacrificing accuracy?
• RQ4 Does the use of geometric clustering reduce memory consumption while preserving fine-grained details lost in downsampling?
• RQ5 How does the depth of the backbone network affect feature quality and model performance in a graph convolutional setting?

Key findings

• MuGNet processes up to 45 room scans (average 2.6M points per scan) on a single 11GB GPU, demonstrating high inference efficiency.
• The model achieves 88.5% overall accuracy and 69.8% mIOU on the S3DIS dataset, outperforming SPG by 3 percentage points in overall accuracy and 7.7 points in mIOU.
• Ablation studies show that increasing backbone depth beyond 14 layers degrades performance due to information assimilation, despite higher memory usage.
• Stacking two Bidirectional GraphConv layers improves mIOU to 66.3%, but the full MuGNet configuration with optimized depth achieves the best result of 69.8% mIOU.
• Removing the bidirectional fusion mechanism results in lower performance, confirming its critical role in enriching feature representations.
• The framework maintains high accuracy while reducing memory demand through pre-clustering, enabling large-scale batch inference on consumer-grade hardware.