[Paper Review] Spherical CNNs

Taco Cohen, Mario Geiger+2 more|arXiv (Cornell University)|Jan 30, 2018

Medical Image Segmentation Techniques34 references124 citations

TL;DR

This paper introduces spherical CNNs by defining a rotation-equivariant spherical cross-correlation using a generalized Fourier theorem and non-commutative FFT, enabling efficient and accurate learning on spherical data. The method achieves state-of-the-art performance in 3D model recognition and atomization energy regression, overcoming distortions from planar projections.

ABSTRACT

Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression.

Motivation & Objective

• To address the limitations of standard CNNs on spherical data due to distortions from planar projections.
• To develop a mathematically sound, rotation-equivariant convolution operation for spherical signals.
• To enable efficient computation of spherical correlations using a generalized Fast Fourier Transform on the rotation group.
• To demonstrate the effectiveness of spherical CNNs in real-world applications like 3D model recognition and molecular energy prediction.

Proposed method

• Proposes a novel spherical cross-correlation operation that is both expressive and rotation-equivariant.
• Derives a generalized Fourier theorem for spherical signals, enabling spectral domain computation.
• Utilizes a non-commutative Fast Fourier Transform (FFT) to compute correlations efficiently in the spectral domain.
• Employs weight sharing across the sphere while preserving rotational invariance through group-theoretic principles.
• Designs a spherical CNN architecture that integrates the spherical convolution layer with standard deep learning components.
• Validates the method using spherical signal processing benchmarks and real-world datasets in 3D vision and quantum chemistry.

Experimental results

Research questions

• RQ1 Can a spherical CNN be constructed to maintain rotational equivariance while enabling efficient computation?
• RQ2 How can spherical cross-correlation be defined to avoid distortions from planar projections?
• RQ3 To what extent does the generalized Fourier theorem on SO(3) enable fast and accurate spherical convolutions?
• RQ4 How does the spherical CNN perform compared to planar CNNs on spherical data in practical applications?

Key findings

• The proposed spherical cross-correlation is rotation-equivariant and enables effective feature learning on spherical manifolds.
• The generalized Fourier theorem allows efficient computation via a non-commutative FFT, reducing computational complexity.
• Spherical CNNs outperform standard CNNs applied to planar projections in 3D model recognition tasks.
• The method achieves high accuracy in atomization energy regression, demonstrating strong performance on molecular property prediction.
• Numerical experiments confirm the method's computational efficiency and numerical stability on spherical grids.