[Paper Review] Convolutions on Spherical Images

Marc Eder, Jan‐Michael Frahm|arXiv (Cornell University)|May 21, 2019

Historical Geography and Cartography13 citations

TL;DR

This paper proposes a novel spherical image representation using the icosahedral Snyder equal-area (ISEA) projection on a geodesic grid to enable efficient and accurate convolutional neural networks on spherical images. By leveraging cartographic principles, the method achieves a 12.6% improvement in semantic segmentation accuracy over state-of-the-art approaches.

ABSTRACT

Applying convolutional neural networks to spherical images requires particular considerations. We look to the millennia of work on cartographic map projections to provide the tools to define an optimal representation of spherical images for the convolution operation. We propose a representation for deep spherical image inference based on the icosahedral Snyder equal-area (ISEA) projection, a projection onto a geodesic grid, and show that it vastly exceeds the state-of-the-art for convolution on spherical images, improving semantic segmentation results by 12.6%.

Motivation & Objective

• To address the challenge of applying convolutional neural networks to spherical images, which lack standard grid structures.
• To develop a representation that preserves geometric and topological properties of spherical data for effective convolution operations.
• To leverage cartographic map projections—specifically the ISEA projection—for optimal spherical image processing.
• To improve semantic segmentation performance on spherical images beyond current state-of-the-art methods.
• To establish a geodesic grid-based framework that enables efficient and accurate deep learning on spherical surfaces.

Proposed method

• The method uses the icosahedral Snyder equal-area (ISEA) projection to map spherical surfaces onto a geodesic grid, minimizing distortion in area and shape.
• It transforms spherical images into a discrete, structured grid representation suitable for standard convolutional operations.
• The geodesic grid ensures uniform sampling and consistent neighborhood relationships across the sphere, avoiding polar singularities.
• Convolutional layers are applied directly on the ISEA-projected grid, enabling end-to-end learning with standard CNN architectures.
• The approach inherits the equal-area property of the ISE projection, preserving spatial fidelity critical for semantic segmentation.
• The framework supports scalable and differentiable processing of spherical images using standard deep learning tools.

Experimental results

Research questions

• RQ1 Can a cartographically sound projection improve the performance of convolutional neural networks on spherical images?
• RQ2 How does the ISEA projection compare to other spherical grid representations in terms of geometric fidelity and CNN performance?
• RQ3 To what extent can a geodesic grid representation enhance semantic segmentation accuracy on spherical data?
• RQ4 Does the use of an equal-area projection reduce distortion-related errors in spherical image processing?
• RQ5 Can a structured, non-rectangular grid support effective and efficient convolution operations in deep learning?

Key findings

• The proposed ISEA-based representation achieves a 12.6% improvement in semantic segmentation accuracy over the previous state of the art.
• The geodesic grid structure eliminates polar singularities present in traditional equirectangular projections.
• The equal-area property of the ISE projection ensures consistent spatial representation across the sphere.
• The method enables direct application of standard convolutional layers on spherical data without significant distortion.
• The approach demonstrates superior performance due to better preservation of local and global geometric relationships.
• The results confirm that cartographic principles can be effectively leveraged to design optimal deep learning representations for spherical images.