[Paper Review] Convolutional Networks for Spherical Signals

Taco Cohen, Mario Geiger+2 more|arXiv (Cornell University)|Sep 14, 2017

Neural Networks and ApplicationsComputer Science56 citations

TL;DR

This paper introduces spherical convolutional networks that perform convolutions on the sphere and SO(3) with rotational weight sharing, enabling rotation-equivariant deep learning on spherical data and demonstrating strong rotation-invariant classification on spherical MNIST.

ABSTRACT

The success of convolutional networks in learning problems involving planar signals such as images is due to their ability to exploit the translation symmetry of the data distribution through weight sharing. Many areas of science and egineering deal with signals with other symmetries, such as rotation invariant data on the sphere. Examples include climate and weather science, astrophysics, and chemistry. In this paper we present spherical convolutional networks. These networks use convolutions on the sphere and rotation group, which results in rotational weight sharing and rotation equivariance. Using a synthetic spherical MNIST dataset, we show that spherical convolutional networks are very effective at dealing with rotationally invariant classification problems.

Motivation & Objective

• Motivate and formalize the use of group equivariant convolutions for spherical data.
• Define spherical and SO(3) convolutions and their Fourier-theoretic foundations.
• Develop an efficient generalized FFT-based implementation for spherical/CovSO(3) convolutions.
• Demonstrate the rotational invariance benefits through experiments on a spherical MNIST dataset.

Proposed method

• Define f * ψ on S^2 as a function on SO(3) using inner products with rotated filters (Eq. 1).
• Define f * ψ on SO(3) via integration over SO(3) (Eq. 2).
• Use generalized Fourier transform (GFT) on S^2 and SO(3) with Wigner D-functions and spherical harmonics (Eq. 3-4).
• Apply convolution theorem in the GFT domain to achieve efficient spherical/SO(3) convolutions.
• Construct a spherical ConvNet: S^2 Conv → nonlinearity → SO(3) Conv → nonlinearities → final linear layer and softmax.
• Utilize bandwidth reduction and channel expansion across layers; enable residuals and batch normalization.

Experimental results

Research questions

• RQ1 Can spherical and SO(3) convolutions provide rotation equivariance across network layers?
• RQ2 How can generalized FFTs be used to efficiently implement convolutions on the sphere and rotation group?
• RQ3 Does a spherical CNN exhibit improved rotation-invariant classification compared to planar CNNs?
• RQ4 What is the numerical behavior of equivariance under sampling and depth?
• RQ5 How does the spherical MNIST benchmark demonstrate rotational robustness of the proposed architecture?

Key findings

• Spherical and SO(3) convolutions are mathematically equivariant to rotations.
• An efficient GFFT-based implementation supports convolution on S^2 and SO(3).
• Spherical CNNs outperform planar CNNs on rotated spherical data in spherical MNIST.
• The spherical CNN maintains high accuracy when training on non-rotated data and testing on rotated data (NR/R).
• On rotated training and testing (R/R), spherical CNN achieves 0.91 accuracy vs. planar 0.45.
• On non-rotated training and rotated testing (NR/R), spherical CNN achieves 0.85 accuracy vs. planar 0.09.