[Paper Review] Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks

Zhiqian Chen, Fanglan Chen+8 more|arXiv (Cornell University)|Feb 27, 2020

Advanced Graph Neural NetworksComputer Science70 references34 citations

TL;DR

The paper presents a unified, theory-driven framework that connects spatial-based and spectral-based GNNs, organizing existing methods into three subcategories per domain and linking their mechanisms through frequency response and aggregation perspectives.

ABSTRACT

Deep learning's success has been widely recognized in a variety of machine learning tasks, including image classification, audio recognition, and natural language processing. As an extension of deep learning beyond these domains, graph neural networks (GNNs) are designed to handle the non-Euclidean graph-structure which is intractable to previous deep learning techniques. Existing GNNs are presented using various techniques, making direct comparison and cross-reference more complex. Although existing studies categorize GNNs into spatial-based and spectral-based techniques, there hasn't been a thorough examination of their relationship. To close this gap, this study presents a single framework that systematically incorporates most GNNs. We organize existing GNNs into spatial and spectral domains, as well as expose the connections within each domain. A review of spectral graph theory and approximation theory builds a strong relationship across the spatial and spectral domains in further investigation.

Motivation & Objective

• Provide a unified framework that connects spatial-domain and spectral-domain GNNs.
• Categorize spatial-based methods by node aggregation type and spectral-based methods by frequency response type.
• Show intra- and inter-domain relationships and establish equivalences between propagation and filtering paradigms.
• Analyze how modern GNN advances (e.g., over-smoothing, scalability) fit within the proposed framework.

Proposed method

• Define a graph neural network framework linking node aggregation f(G)X in the spatial domain to frequency response g(Λ) applied in the spectral domain.
• Categorize spatial methods into linear, polynomial, and rational propagation (A-1, A-2, A-3) and spectral methods into linear, polynomial, and rational approximation (B-1, B-2, B-3).
• Establish equivalence relationships between A-0 and B-0 and show generalization/specialization links across subcategories.
• Translate representative models (GCN, GraphSAGE, GIN, ChebNet, DCNN, SGC, ARMA, PPNP, LP) into the unified framework to illustrate correspondences.
• Discuss time complexity and expressive power, and relate sampling and over-smoothing techniques to the framework.

Experimental results

Research questions

• RQ1 How can spatial-based and spectral-based GNNs be unified under a single theoretical framework?
• RQ2 What are the exact correspondences and equivalences between common spatial propagation schemes and spectral filtering approaches?
• RQ3 How do modern GNN techniques for scalability and over-smoothing fit within the unified framework?
• RQ4 What are the trade-offs between expressive power and computational efficiency across the unified categories?
• RQ5 Can reverse propagation and rational approximations be interpreted as generalizations of standard neighborhood aggregation?

Key findings

• A unified taxonomy maps spatial (A-1/A-2/A-3) and spectral (B-1/B-2/B-3) methods and shows explicit links between aggregation and frequency response.
• Many classic GNNs (e.g., GCN, GraphSAGE, GIN) correspond to specific instances within the linear propagation category and have clear spectral equivalents.
• Polynomial and rational propagation provide higher expressive power at the cost of increased computation, with rational methods offering the strongest approximation capabilities.
• The framework reveals that over-smoothing and scaling issues are special cases of the proposed A-0/B-0 framework and can be analyzed via approximation theory.
• Sampling methods (random walks, subgraph sampling) and deep architectures for scalability align with A-1/A-2 categories, while sampling does not belong to A-3 or B-3 in the cited discussions.
• There is a trade-off between expressive power and efficiency: A-3/B-3 converge faster on challenging signals but incur higher overhead.