[Paper Review] Incidence Networks for Geometric Deep Learning

Marjan Albooyeh|arXiv (Cornell University)|Jan 1, 2019

Topological and Geometric Data Analysis39 references11 citations

TL;DR

This paper introduces Incidence Networks, a family of equivariant deep learning models for structured geometric data like graphs, simplicial complexes, and polytopes using incidence tensors. It shows that equivariant linear maps on these tensors decompose into efficient pooling-and-broadcasting operations, reducing parameter count while preserving symmetry, with a key result that such maps can be systematically constructed via face-vector decomposition under symmetric group actions.

ABSTRACT

Sparse incidence tensors can represent a variety of structured data. For example, we may represent attributed graphs using their node-node, node-edge, or edge-edge incidence matrices. In higher dimensions, incidence tensors can represent simplicial complexes and polytopes. In this work, we formalize incidence tensors, analyze their structure, and present the family of equivariant networks that operate on them. We show that any incidence tensor decomposes into invariant subsets. This decomposition, in turn, leads to a decomposition of the corresponding equivariant layer that allows efficient and intuitive pooling-and-broadcasting implementation, for both dense and sparse tensors. We demonstrate the effectiveness of this family of networks by reporting state-of-the-art on graph learning tasks for many targets in the QM9 dataset.

Motivation & Objective

• To formalize incidence tensors as a unified representation for structured geometric data such as graphs, simplicial complexes, and polytopes.
• To design equivariant deep learning models that respect the symmetry of geometric structures, particularly under the symmetric group action.
• To decompose equivariant linear maps on incidence tensors into efficient, parameter-efficient pooling-and-broadcasting operations.
• To unify and generalize existing equivariant models, such as those in Maron et al. (2018), by showing they are special cases of the proposed framework.
• To enable efficient, modular, and scalable geometric deep learning through face-vector decomposition and orbit-based parameter sharing.

Proposed method

• Represent geometric data using sparse incidence tensors that encode incidence patterns between faces of different dimensions.
• Use the symmetric group SN as a global symmetry group, enabling equivariance across diverse geometric instances despite varying automorphism groups.
• Decompose incidence tensors into invariant subsets (orbits) corresponding to faces of fixed size, forming face-vectors for each dimension.
• Characterize equivariant linear maps as linear combinations of pooling-and-broadcasting operations between face-vectors of different sizes.
• Derive the number of independent parameters using the formula τm→m′ = ∑m=0min(M,M′) (M choose m)(M′ choose m)m!, which counts valid pooling-and-broadcasting configurations.
• Construct Incidence Networks by stacking such equivariant layers, with parameters shared across equivalent structural patterns via face-vector decomposition.

Experimental results

Research questions

• RQ1 How can incidence tensors be systematically used to represent diverse geometric data structures like graphs, simplicial complexes, and polytopes?
• RQ2 What is the structure of equivariant linear maps on incidence tensors under the symmetric group action, and how can they be decomposed?
• RQ3 Can equivariant models for geometric data be made both efficient and generalizable by replacing instance-specific automorphism groups with a global symmetric group?
• RQ4 What is the minimal set of operations needed to construct all equivariant linear maps on incidence tensors, and how can they be parameterized efficiently?
• RQ5 How do existing equivariant models, such as those in Maron et al. (2018), fit into this unified framework of pooling-and-broadcasting operations?

Key findings

• Any incidence tensor decomposes into invariant orbits under the symmetric group action, corresponding to faces of fixed size, enabling a face-vector representation.
• Equivariant linear maps on incidence tensors decompose into efficient pooling-and-broadcasting operations, with the number of such operations given by τm→m′ = ∑m=0min(M,M′) (M choose m)(M′ choose m)m!.
• The total number of independent parameters in a layer is τ = ∑m,m′ κ′m′κmτm→m′, which is significantly smaller than naive parameterization.
• The framework generalizes and unifies existing equivariant models, such as those in Maron et al. (2018), showing they are special cases of the proposed pooling-and-broadcasting construction.
• For a D-order node-node-...-node tensor, the total number of parameters is Bell(2D), matching the result from Maron et al. (2018), validating the framework's consistency.
• The model achieves parameter efficiency by replacing dense weight matrices with structured, sparsity-aware pooling-and-broadcasting operations, reducing complexity while preserving equivariance.