[Paper Review] Spherical CNNs on Unstructured Grids

Chiyu Max Jiang, Jingwei Huang+4 more|arXiv (Cornell University)|Jan 7, 2019

Remote Sensing and LiDAR Applications66 citations

TL;DR

The paper introduces MeshConv, a parameterized differential operator-based convolution for meshes, enabling efficient orientable CNNs on unstructured grids for spherical signals with fewer parameters while matching or surpassing state-of-the-art performance across tasks.

ABSTRACT

We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals. To this end, we replace conventional convolution kernels with linear combinations of differential operators that are weighted by learnable parameters. Differential operators can be efficiently estimated on unstructured grids using one-ring neighbors, and learnable parameters can be optimized through standard back-propagation. As a result, we obtain extremely efficient neural networks that match or outperform state-of-the-art network architectures in terms of performance but with a significantly lower number of network parameters. We evaluate our algorithm in an extensive series of experiments on a variety of computer vision and climate science tasks, including shape classification, climate pattern segmentation, and omnidirectional image semantic segmentation. Overall, we present (1) a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals, and (2) we show that our unique kernel parameterization allows our model to achieve the same or higher accuracy with significantly fewer network parameters.

Motivation & Objective

• Develop orientable CNNs on unstructured grids (meshes) for spherical signals.
• Propose a Kernel parameterization using differential operators to reduce parameters.
• Show that the approach achieves high accuracy with fewer parameters across tasks.
• Demonstrate applicability to spherical MNIST, 3D object classification, omnidirectional segmentation, and climate pattern segmentation.
• Open-source code release for further research and extension.

Proposed method

• Replace conventional CNN kernels with a linear combination of differential operators parameterized by learnable weights.
• Use four operators: identity, first-order derivatives in x and y, and Laplacian to form the kernel (theta0*I + theta1*dx + theta2*dy + theta3*L).
• Estimate differential operators efficiently on unstructured meshes via one-ring neighborhoods and cotangent-based discretization for the Laplacian.
• Discretize the sphere with an icosahedral mesh to enable easy coarsening/refinement and pooling/unpooling.
• Train end-to-end with backpropagation and standard optimizers.
• Provide MeshConv architecture with encoder–decoder variants for classification, segmentation, and climate tasks.

Experimental results

Research questions

• RQ1 Can parameterized differential operators (PDOs) approximate conventional convolution kernels on unstructured grids with fewer parameters?
• RQ2 Do PDO-based MeshConv networks achieve competitive or superior accuracy on spherical and manifold-valued tasks while improving parameter efficiency?
• RQ3 How well do orientable spherical CNNs perform on tasks like omnidirectional image segmentation and climate pattern segmentation compared to state-of-the-art methods?
• RQ4 Is an icosahedral spherical mesh an effective discretization for CNNs on spherical signals in terms of accuracy and efficiency?
• RQ5 What is the impact of including multiple differential operators in the kernel (ablation study) on performance?

Key findings

• MeshConv with PDOs achieves state-of-the-art or competitive accuracy with significantly fewer parameters across tasks.
• On spherical MNIST, the proposed method achieves 99.23% accuracy with about 62k parameters, outperforming S2CNN and SphereNet baselines.
• ModelNet40 results show competitive peak accuracy with superior parameter efficiency compared to PointNet++, VoxNet, and S2CNN baselines.
• Omnidirectional image segmentation on 2D3DS demonstrates that spherical segmentation outperforms planar baselines and 3D point-based methods across parameter regimes.
• Climate pattern segmentation (AR and TC) using full-global inputs yields higher mean accuracy than the baseline that uses random crops, illustrating holistic understanding from global data.
• Ablation study confirms that including all differential operator components provides best accuracy, with performance improving as more operators are added.