[Paper Review] The geometry of rank decompositions of matrix multiplication II: $3 imes 3$ matrices

Grey Ballard, Christian Ikenmeyer+2 more|arXiv (Cornell University)|Jan 2, 2018

Tensor decomposition and applications6 references3 citations

TL;DR

This paper presents new rank 23 decompositions of the $3\times3$ matrix multiplication tensor $M_{\langle 3\rangle}$ with enhanced symmetry, including a novel decomposition with 11 cubes and a symmetry group of order 12. The authors develop methods to identify symmetry groups and equivalence classes of decompositions, leveraging representation theory and invariant theory to guide the search for structured, symmetric rank decompositions in higher dimensions.

ABSTRACT

This is the second in a series of papers on rank decompositions of the matrix multiplication tensor. We present new rank $23$ decompositions for the $3 imes 3$ matrix multiplication tensor $M_{\langle 3 angle}$. All our decompositions have symmetry groups that include the standard cyclic permutation of factors but otherwise exhibit a range of behavior. One of them has 11 cubes as summands and admits an unexpected symmetry group of order 12. We establish basic information regarding symmetry groups of decompositions and outline two approaches for finding new rank decompositions of $M_{\langle n angle}$ for larger $n$.

Motivation & Objective

• To discover new rank 23 decompositions of the $3\times3$ matrix multiplication tensor $M_{\langle 3\rangle}$ that respect the standard cyclic $\mathbb{Z}_3$-symmetry.
• To develop systematic techniques for identifying the symmetry groups of rank decompositions and determining when two decompositions are equivalent.
• To establish a framework using representation theory to construct new rank decompositions for $M_{\langle \mathbf{n} \rangle}$ for larger $\mathbf{n}$, guided by symmetry.
• To explore the geometric and algebraic structure of $\mathbb{Z}_{n+1}$-invariant subspaces in the tensor space $ (\mathbb{C}^{n^2})^{{\otimes}3} $, particularly for $n=3$.
• To investigate the role of symmetry in minimizing tensor rank and to explain why numerical methods may fail to find optimal decompositions due to convergence to border rank solutions.

Proposed method

• Employ algebraic geometry and representation theory to analyze the decomposition space of $M_{\langle 3\rangle}$, focusing on $\mathbb{Z}_3$-invariant components.
• Use eigenvalue analysis and characteristic polynomials of associated linear operators to distinguish between decomposition families and detect symmetries.
• Apply invariant theory to decompose the tensor space $ (\mathbb{C}^{n^2})^{{\otimes}3} $ into irreducible representations under the action of $\mathbb{Z}_{n+1} \times \mathbb{Z}_3$, computing dimensions of invariant subspaces.
• Implement numerical search methods (e.g., alternating least squares) with symmetry constraints to reduce search space and guide convergence toward symmetric rank decompositions.
• Utilize projective geometry to interpret decompositions as configurations of points in $\mathbb{P}^{n-1}$, enabling geometric comparison of different decompositions.
• Construct explicit decompositions by combining symmetric building blocks such as symmetric and antisymmetric tensors, and verify their correctness via trace identities.

Experimental results

Research questions

• RQ1 What are the symmetry groups of rank 23 decompositions of $M_{\langle 3\rangle}$, and how do they relate to the standard cyclic $\mathbb{Z}_3$-symmetry?
• RQ2 Can new rank 23 decompositions of $M_{\langle 3\rangle}$ be constructed that are invariant under larger symmetry groups than previously known?
• RQ3 How can the equivalence of two rank decompositions be determined algebraically, especially when they differ only by coordinate transformations?
• RQ4 What is the structure of the space of $\mathbb{Z}_{n+1} \times \mathbb{Z}_3$-invariant tensors in $ (\mathbb{C}^{n^2})^{{\otimes}3} $, and how does it vary with $n$?
• RQ5 Why do numerical methods for tensor decomposition often converge to border rank 22 solutions rather than rank 22 decompositions, and how can symmetry help overcome this?

Key findings

• The authors present three new rank 23 decompositions of $M_{\langle 3\rangle}$, all invariant under the standard cyclic $\mathbb{Z}_3$-symmetry, resolving a gap in prior work.
• One decomposition contains 11 cubic terms and admits a symmetry group of order 12, an unexpected and significant enhancement over previously known symmetric decompositions.
• The space of $\mathbb{Z}_4 \times \mathbb{Z}_3$-invariant tensors in $ (\mathbb{C}^9)^{{\otimes}3} $ has dimension 10, while the $\mathbb{Z}_4 \times \mathbb{Z}_3$-invariant subspace for $n=3$ has dimension 63.
• For $n=4$, the $\mathbb{Z}_5 \times \mathbb{Z}_3$-invariant subspace in $ (\mathbb{C}^{16})^{{\otimes}3} $ has dimension 276, significantly smaller than the full space of dimension 4096.
• The authors identify three distinct families of decompositions via characteristic polynomials of associated matrices, with counts of 6, 4, and 1 for symmetric, cyclic, and triple eigenvalue types.
• The study confirms that symmetry is a powerful guide in tensor decomposition: imposing $\mathbb{Z}_3$-invariance reduces search space and increases likelihood of finding optimal or near-optimal decompositions.