[Paper Review] Clifford Group Equivariant Neural Networks

David Ruhe, J. Brandstetter+1 more|arXiv (Cornell University)|May 18, 2023

Advanced Neuroimaging Techniques and Applications9 citations

TL;DR

This paper introduces Clifford Group Equivariant Neural Networks (CGENNs) that achieve O(n) and E(n) equivariance by leveraging the Clifford group acting on the entire Clifford algebra, enabling polynomial, grade-projection, and geometric-product based neural layers that generalize across dimensions.

ABSTRACT

We introduce Clifford Group Equivariant Neural Networks: a novel approach for constructing $\mathrm{O}(n)$- and $\mathrm{E}(n)$-equivariant models. We identify and study the $ extit{Clifford group}$, a subgroup inside the Clifford algebra tailored to achieve several favorable properties. Primarily, the group's action forms an orthogonal automorphism that extends beyond the typical vector space to the entire Clifford algebra while respecting the multivector grading. This leads to several non-equivalent subrepresentations corresponding to the multivector decomposition. Furthermore, we prove that the action respects not just the vector space structure of the Clifford algebra but also its multiplicative structure, i.e., the geometric product. These findings imply that every polynomial in multivectors, An advantage worth mentioning is that we obtain expressive layers that can elegantly generalize to inner-product spaces of any dimension. We demonstrate, notably from a single core implementation, state-of-the-art performance on several distinct tasks, including a three-dimensional $n$-body experiment, a four-dimensional Lorentz-equivariant high-energy physics experiment, and a five-dimensional convex hull experiment.

Motivation & Objective

• Motivate the use of symmetry and equivariance in neural networks for geometric and physical problems.
• Develop a Clifford-algebra based framework that yields equivariant maps under the Clifford group.
• Show that polynomials and grade projections are Clifford group equivariant and can be parameterized in neural layers.
• Construct and evaluate CGENN layers (linear, geometric-product, and gated nonlinearities) that respect the Clifford group action.
• Demonstrate generalization to arbitrary dimensions and metric signatures, enabling O(n) and E(n) equivariance.

Proposed method

• Define Cl(V,q) and its grade decomposition into Cl^(m)(V,q) with dim Cl^(m)=binomial(n,m).
• Introduce the Clifford group Γ(V,q) and the adjusted twisted conjugation ρ(w) that acts as an algebra automorphism.
• Prove that ρ(w) preserves grades and the extended quadratic form, making grade projections and polynomials equivariant.
• Parameterize neural layers using multivector inputs in a steerable basis, enabling linear and geometric-product based transformations.
• Use normalization and tailored nonlinearities to maintain numerical stability while preserving equivariance.
• Embed data into Cl(V,q) via scalars and vectors, and produce outputs through grade-projected predictions (scalars or vectors).

Experimental results

Research questions

• RQ1 Can Clifford-group actions on the full Clifford algebra yield practical O(n) or E(n) equivariant neural networks without resorting to spherical-harmonics bases or Clebsch–Gordan coefficients?
• RQ2 Do polynomials in multivectors and grade-projections provide a dense, expressive, and dimension-agnostic parameterization for equivariant layers?
• RQ3 How do CGENN layers compare to existing equivariant architectures on tasks requiring geometric and physical invariances (e.g., n-body dynamics, Lorentz-equivariant physics, high-dimensional convex hulls)?
• RQ4 What are effective layer designs (linear, geometric-product, and fully-connected product layers) and normalization schemes that preserve Clifford-group equivariance in practice?

Key findings

• CGENNs achieve state-of-the-art performance on diverse tasks, including 3D n-body dynamics, 4D Lorentz-equivariant high-energy physics, and 5D convex hull problems.
• All grade projections are Clifford-group equivariant, enabling dense and expressive parameterizations via polynomials in multivectors.
• The Clifford-group action extends naturally to the full Clifford algebra, yielding orthogonal representations on each grade and bypassing explicit tensor-product constructions.
• The framework generalizes to O(n) and E(n) equivariance across arbitrary dimensions and metric signatures, without requiring Clebsch–Gordan coefficients or alternative bases.
• Experimental results show CGENNs outperform scalarization baselines in volume-related tasks, with some data-efficiency considerations in very low-data regimes.
• The approach enables modeling of exotic geometric objects (e.g., pseudovectors) and preserves multiplicative structure through the geometric product, enriching the representation capacity.