[Paper Review] On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups

Risi Kondor, Shubhendu Trivedi|arXiv (Cornell University)|Feb 11, 2018

Advanced Graph Neural Networks20 references215 citations

TL;DR

The paper proves that, under natural conditions, equivariance to a compact group action in neural networks is equivalent to each layer implementing a generalized convolution derived from the group operation and Haar measure; it provides a rigorous framework linking equivariance and convolution beyond translations.

ABSTRACT

Convolutional neural networks have been extremely successful in the image recognition domain because they ensure equivariance to translations. There have been many recent attempts to generalize this framework to other domains, including graphs and data lying on manifolds. In this paper we give a rigorous, theoretical treatment of convolution and equivariance in neural networks with respect to not just translations, but the action of any compact group. Our main result is to prove that (given some natural constraints) convolutional structure is not just a sufficient, but also a necessary condition for equivariance to the action of a compact group. Our exposition makes use of concepts from representation theory and noncommutative harmonic analysis and derives new generalized convolution formulae.

Motivation & Objective

• Motivate extending CNN-style equivariance beyond translations to general compact group actions.
• Provide a rigorous, representation-theoretic framework unifying convolution and equivariance.
• Show that generalized convolutions on groups and quotient spaces are necessary and sufficient for G-equivariance in feed-forward nets.
• Develop and present generalized convolution formulas for functions on groups and homogeneous spaces.
• Highlight implications for architectures operating on data with symmetries beyond Euclidean translations.

Proposed method

• Define generalized convolution on finite or countable groups and on quotient spaces using the group operation and Haar measure.
• Formalize G-equivariant feed-forward networks where each layer implements a generalized convolution.
• Use lifting and projection between G and homogeneous spaces to handle activations defined on quotient spaces.
• Connect convolution on groups to its Fourier-space representation via irreducible representations (harmonic analysis).
• Derive special cases of convolution corresponding to X = G, Y = G/H, and X = G/H, Y = Hackslash G, including mixed cases with double cosets (G/K).
• Provide abstract theory first, with concrete examples in Section 6.

Experimental results

Research questions

• RQ1 Under what conditions is a feed-forward network equivariant to the action of a compact group G?
• RQ2 How can convolution be generalized to act on functions defined on quotient/homogeneous spaces derived from G?
• RQ3 What is the relationship between equivariance and generalized convolution in networks operating on G or its homogeneous spaces?
• RQ4 How does Fourier analysis (representations) describe generalized convolutions on these spaces?
• RQ5 What are the practical implications for architectures handling data with symmetries beyond translations (graphs, manifolds, etc.)?

Key findings

• Equivariance to a compact group G in a feed-forward network is equivalent to each layer implementing a generalized convolution derived from (1) with respect to G.
• Convolution extends naturally to quotient spaces and homogeneous spaces, yielding well-defined operations on G/H, Hackslash G, and G/K.
• The Fourier transform of functions on quotient spaces has specific sparsity patterns determined by how representations decompose on subgroups H and K.
• Three main convolution cases are analyzed: X = G, Y = G/H; X = G/H, Y = Hackslash G; and X = G/H, Y = Hackslash G/K, each producing outputs on corresponding quotient spaces.
• Convolution on quotient spaces connects to representation theory, providing a framework to analyze and design G-equivariant neural networks.