[Paper Review] General $E(2)$-Equivariant Steerable CNNs

Maurice Weiler, Gabriele Cesa|arXiv (Cornell University)|Nov 19, 2019

Geophysical Methods and Applications105 citations

TL;DR

This paper provides a general framework for E(2)-equivariant steerable convolutions on the plane, reduces kernel constraints to irreducible representations, and demonstrates broad architectural applicability with empirical gains.

ABSTRACT

The big empirical success of group equivariant networks has led in recent years to the sprouting of a great variety of equivariant network architectures. A particular focus has thereby been on rotation and reflection equivariant CNNs for planar images. Here we give a general description of $E(2)$-equivariant convolutions in the framework of Steerable CNNs. The theory of Steerable CNNs thereby yields constraints on the convolution kernels which depend on group representations describing the transformation laws of feature spaces. We show that these constraints for arbitrary group representations can be reduced to constraints under irreducible representations. A general solution of the kernel space constraint is given for arbitrary representations of the Euclidean group $E(2)$ and its subgroups. We implement a wide range of previously proposed and entirely new equivariant network architectures and extensively compare their performances. $E(2)$-steerable convolutions are further shown to yield remarkable gains on CIFAR-10, CIFAR-100 and STL-10 when used as a drop-in replacement for non-equivariant convolutions.

Motivation & Objective

• Motivate the use of symmetry priors to improve generalization and sample efficiency in planar image networks.
• Provide a general strategy to solve kernel constraints for E(2) and its subgroups by reducing to irreducible representations.
• Enable a unified framework that encompasses and extends prior GCNN, Steerable CNN, Harmonic, and related architectures.
• Showhow group representations and nonlinearities interact to constrain and enable various equivariant layers.
• Demonstrate practical benefits across datasets by treating E(2)-steerable convolutions as drop-in replacements for standard convolutions.

Proposed method

• Formulate steerable feature fields with transformation laws defined by group representations and induced representations.
• Derive a kernel constraint k(gx) = ρ_out(g) k(x) ρ_in(g^{-1}) and show it can be solved via irrep decomposition (Eq. 3).
• Decompose input/output representations into irreducibles to obtain independent irrep blocks for kernel constraints (Eq. 3).
• Expand kernels angularly using Fourier bases to exploit O(2) and subgroup irreps, yielding explicit basis elements (Table 2 and Appendix F).
• Construct basis-based G-steerable kernels and learn linear combinations to parameterize equivariant convolutions (Eq. 4).
• Discuss and evaluate various representations (regular, quotient, induced) and compatible nonlinearities under unitary constraints.

Experimental results

Research questions

• RQ1 How can the kernel space constraint for E(2)-equivariant convolutions be efficiently solved for arbitrary representations?
• RQ2 What are the explicit bases of steerable kernels for O(2) and its subgroups, and how do irreps determine allowable angular components?
• RQ3 How do different representations and nonlinearities interact to influence network performance and parameter efficiency?
• RQ4 Can group restriction (progressive reduction of equivariance) improve performance on real images by aligning with data symmetry?
• RQ5 Do E(2)-steerable convolutions provide consistent gains as drop-in replacements across standard benchmarks (CIFAR-10/100, STL-10, MNIST variants)?

Key findings

• A general solution to the kernel constraint is achievable by reducing to irreducible representations, enabling a unified framework for many prior architectures.
• Explicit angular bases are derived for G-steerable kernels corresponding to O(2) and its subgroups, with bases associated to distinct angular frequencies.
• The framework supports hybrid architectures combining different field types and allows for group restriction operations to vary equivariance with depth.
• Empirical benchmarks compare groups, representations, and nonlinearities, showing significant performance gains when replacing conventional convolutions with E(2)-steerable convolutions on CIFAR-10, CIFAR-100, and STL-10.
• The approach is transferable to other homogeneous spaces and manifolds, enabling extensions like spherical CNNs and gauge-equivariant CNNs.
• The design accommodates regular and quotient representations and shows how nonlinearities must be chosen to maintain equivariance.