[Paper Review] Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks

Zhiqian Chen, Fanglan Chen+8 more|arXiv (Cornell University)|Jul 21, 2021

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes a unified theoretical framework that bridges spatial and spectral graph neural networks (GNNs) using approximation theory and rational functions, demonstrating that diverse GNN architectures—previously seen as disjoint—can be systematically connected through a common mathematical foundation, enabling deeper theoretical understanding and improved model interpretability.

ABSTRACT

Deep learning's performance has been extensively recognized recently. Graph neural networks (GNNs) are designed to deal with graph-structural data that classical deep learning does not easily manage. Since most GNNs were created using distinct theories, direct comparisons are impossible. Prior research has primarily concentrated on categorizing existing models, with little attention paid to their intrinsic connections. The purpose of this study is to establish a unified framework that integrates GNNs based on spectral graph and approximation theory. The framework incorporates a strong integration between spatial- and spectral-based GNNs while tightly associating approaches that exist within each respective domain.

Motivation & Objective

• To address the lack of a unifying theoretical framework connecting spatial and spectral GNNs, which currently exist as isolated paradigms.
• To establish a common mathematical language based on approximation theory and rational functions that unifies diverse GNN architectures.
• To clarify the intrinsic relationships between seemingly disparate GNN methods, including those based on message passing, spectral filtering, and attention mechanisms.
• To extend the theoretical foundation of GNNs to include challenging graph types such as directed, dynamic, higher-order, and multilayer networks.
• To provide a principled basis for understanding GNN behavior, reducing their black-box nature and enabling more reliable deployment in high-stakes applications.

Proposed method

• Formalizing GNNs through rational function approximation of graph filters, enabling a unified representation of both spatial and spectral methods.
• Using partial differential equations (PDEs), such as diffusion and wave equations, to model the connection between polynomial (spectral) and rational (spatial) filters.
• Introducing a generalized graph convolution framework based on rational functions that subsumes existing GNNs like GCN, GAT, and GraphSAGE.
• Extending the framework to handle directed graphs by leveraging Hermitian transpose and non-symmetric graph Laplacians.
• Applying spectral analysis to model dynamic graph evolution, using eigenvalues and eigenvectors to capture temporal and structural patterns.
• Integrating higher-order interactions via hypergraphs and simplicial complexes, and modeling multilayer networks through supra-adjacency matrices.

Experimental results

Research questions

• RQ1 How can spatial and spectral GNNs be formally connected through a common theoretical framework?
• RQ2 What role do rational functions play in unifying different GNN architectures, and why are they more expressive than polynomial approximations?
• RQ3 Can the PDE-based perspective (e.g., diffusion and wave equations) unify the understanding of graph filtering in both domains?
• RQ4 How can the framework be extended to model directed, dynamic, and multilayer graphs beyond standard undirected, static graphs?
• RQ5 To what extent can this unified framework improve model interpretability and reduce the black-box nature of GNNs?

Key findings

• Theoretical analysis shows that rational functions provide a more expressive and stable approximation of graph filters than polynomial filters, explaining the superior performance of many modern GNNs.
• Spatial and spectral GNNs are shown to be special cases of a broader class of rational graph filters, revealing their underlying mathematical unity.
• The PDE-based interpretation links diffusion (polynomial filters) and wave propagation (rational filters), offering a physical intuition for GNN dynamics.
• The framework enables the modeling of directed graphs via non-symmetric adjacency matrices and Hermitian transpose, overcoming limitations of symmetric matrix assumptions.
• Spectral analysis of dynamic graphs reveals that eigenstructure evolution correlates with structural changes, enabling trajectory prediction and anomaly detection.
• Higher-order interactions and multilayer networks can be systematically modeled using hypergraphs and supra-adjacency matrices, extending the framework’s applicability to complex systems.