[Paper Review] Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks

Mathilde Papillon, Sophia Sanborn+2 more|arXiv (Cornell University)|Apr 20, 2023

Gene Regulatory Network Analysis15 citations

TL;DR

A comprehensive survey introducing Topological Deep Learning and unifying a wide range of message-passing Topological Neural Networks (TNNs) across domains like hypergraphs, simplicial and cellular complexes, and combinatorial complexes. It provides unified notation, graphical tensor diagrams, and discussion of challenges and opportunities.

ABSTRACT

The natural world is full of complex systems characterized by intricate relations between their components: from social interactions between individuals in a social network to electrostatic interactions between atoms in a protein. Topological Deep Learning (TDL) provides a comprehensive framework to process and extract knowledge from data associated with these systems, such as predicting the social community to which an individual belongs or predicting whether a protein can be a reasonable target for drug development. TDL has demonstrated theoretical and practical advantages that hold the promise of breaking ground in the applied sciences and beyond. However, the rapid growth of the TDL literature for relational systems has also led to a lack of unification in notation and language across message-passing Topological Neural Network (TNN) architectures. This presents a real obstacle for building upon existing works and for deploying message-passing TNNs to new real-world problems. To address this issue, we provide an accessible introduction to TDL for relational systems, and compare the recently published message-passing TNNs using a unified mathematical and graphical notation. Through an intuitive and critical review of the emerging field of TDL, we extract valuable insights into current challenges and exciting opportunities for future development.

Motivation & Objective

• Provide an accessible introduction to Topological Deep Learning (TDL) for relational systems.
• Unify and compare published message-passing Topological Neural Networks (TNNs) using a common mathematical and graphical notation.
• Analyze architectures, applications, and practical considerations of TNNs.
• Identify current challenges and opportunities for future development in TDL.

Proposed method

• Present data on discrete domains (graphs, hypergraphs, simplicial complexes, cellular complexes, combinatorial complexes) and their ranks and cells.
• Define boundary relations and incidence matrices to encode neighborhood structures.
• Decompose message passing into four steps (Message, Within-neighborhood aggregation, Between-neighborhood aggregation, Update) and describe how features are updated across layers.
• Use tensor diagrams to graphically represent message passing schemes and compare architectures.
• Classify and review TNN architectures by domain and type of message passing, with unified equations in their notation.

Figure 1: Topological Neural Network : Data associated with a complex system are features defined on a data domain , which is preprocessed into a computational domain that encodes interactions between the system’s components with neighborhoods . The TNN’s layers use message passing to successively u

Experimental results

Research questions

• RQ1 What are the common architectures for message-passing in Topological Neural Networks across different topological domains?
• RQ2 How can a unified notation help compare Hypergraphs, Simplicial Complexes, Cellular Complexes, and Combinatorial Complexes in TNNs?
• RQ3 What are the main strengths and limitations of TNNs for various relational data tasks?
• RQ4 What challenges remain and what opportunities exist for future development in Topological Deep Learning?

Key findings

• Topological Deep Learning extends beyond graphs to domains like hypergraphs, simplicial and cellular complexes, enabling richer modeling of higher-order relations.
• A unified four-step message-passing framework (Message, Within-neighborhood aggregation, Between-neighborhood aggregation, Update) facilitates systematic comparison of TNN architectures across domains.
• Tensor diagrams provide a graphical, intuitive representation of how messages propagate through topological domains and layers.
• TNN architectures vary by domain and by whether they use standard, attentional, or general message passing, with multiple neighborhood structures employed per model.
• The survey highlights that preserving topological structure during learning often yields superior expressivity and performance, though domain-specific limitations and practical considerations remain.

Figure 2: Domains: Nodes in blue, (hyper)edges in pink, and faces in dark red. Figure adapted from [ 11 ] .