[Paper Review] Bridging the Gap Between Spectral and Spatial Domains in Graph Neural Networks

Muhammet Balcılar, Guillaume Renton+4 more|arXiv (Cornell University)|Mar 26, 2020

Advanced Graph Neural Networks27 references38 citations

TL;DR

The paper proves the equivalence of spectral and spatial graph convolutions with a fixed spectral profile, analyzes four popular ConvGNNs from a spectral viewpoint, and introduces depthwise separable spectral designs with transferable filter coefficients.

ABSTRACT

This paper aims at revisiting Graph Convolutional Neural Networks by bridging the gap between spectral and spatial design of graph convolutions. We theoretically demonstrate some equivalence of the graph convolution process regardless it is designed in the spatial or the spectral domain. The obtained general framework allows to lead a spectral analysis of the most popular ConvGNNs, explaining their performance and showing their limits. Moreover, the proposed framework is used to design new convolutions in spectral domain with a custom frequency profile while applying them in the spatial domain. We also propose a generalization of the depthwise separable convolution framework for graph convolutional networks, what allows to decrease the total number of trainable parameters by keeping the capacity of the model. To the best of our knowledge, such a framework has never been used in the GNNs literature. Our proposals are evaluated on both transductive and inductive graph learning problems. Obtained results show the relevance of the proposed method and provide one of the first experimental evidence of transferability of spectral filter coefficients from one graph to another. Our source codes are publicly available at: https://github.com/balcilar/Spectral-Designed-Graph-Convolutions

Motivation & Objective

• Bridge the gap between spectral and spatial designs for graph convolutions.
• Provide a spectral analysis of popular ConvGNNs to explain performance and limitations.
• Design new spectral-domain convolutions with custom frequency profiles applied in the spatial domain.
• Generalize to a depthwise separable convolution framework to reduce parameters while maintaining capacity.
• Demonstrate transferability of spectral filter coefficients across graphs in inductive settings.

Proposed method

• Theoretical demonstration that spectral ConvGNNs with fixed frequency profile matrix B are a special case of spatial ConvGNNs with kernels C^(s).
• Derivation showing H^(l+1) can be represented as a sum of C^(s) H^(l) W^(l,s) with C^(s) = U diag(digamma_s(lambda)) U^T.
• Spectral analysis of ChebNet, CayleyNet, GCN, and GAT via frequency-profile corollaries to reveal their spectral characteristics.
• Proposal of depthwise separable graph convolutions that use multiple spectral frequency profiles to form a richer, parameter-efficient foundation.
• Discussion of transferability of spectral filter coefficients across different graphs, including inductive learning scenarios.

Experimental results

Research questions

• RQ1 Can spectral-domain parameterizations be reformulated as equivalent spatial-domain graph convolutions?
• RQ2 What are the spectral frequency profiles of four popular ConvGNNs (ChebNet, CayleyNet, GCN, GAT) on representative graphs?
• RQ3 Can we design spectral-domain convolutions with custom frequency profiles and implement them in the spatial domain?
• RQ4 Does a depthwise separable spectral convolution framework reduce parameters without sacrificing model capacity?
• RQ5 Are spectral filter coefficients transferable across graphs in inductive learning settings?

Key findings

• Spectral ConvGNNs with a fixed frequency-profile design are equivalent to spatial ConvGNNs, enabling spectral analysis without Fourier transforms.
• ChebNet, CayleyNet, GCN, and GAT exhibit characteristic frequency profiles, revealing they are largely low-pass and problem-specific rather than universally agnostic.
• GCN shows transferable kernels across different graphs, with a consistent low-pass shape and a calculable cut-off frequency.
• Depthwise separable graph convolutions with multiple spectral profiles can reduce trainable parameters while maintaining model capacity.
• Empirical results on transductive and inductive tasks provide evidence for the transferability of spectral filter coefficients between graphs.