[Paper Review] HoloNets: Spectral Convolutions do extend to Directed Graphs

Christian Koke, Daniel Cremers|arXiv (Cornell University)|Oct 3, 2023

Advanced Graph Neural Networks4 citations

TL;DR

This paper introduces HoloNets, a novel framework that extends spectral convolutions to directed graphs by abandoning reliance on the graph Fourier transform. Using tools from complex analysis and spectral theory, it formulates learnable filters via resolvent-based operators on directed graphs, achieving state-of-the-art performance in heterophilic node classification and robustness to topological perturbations.

ABSTRACT

Within the graph learning community, conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs: Only there could the existence of a well-defined graph Fourier transform be guaranteed, so that information may be translated between spatial- and spectral domains. Here we show this traditional reliance on the graph Fourier transform to be superfluous and -- making use of certain advanced tools from complex analysis and spectral theory -- extend spectral convolutions to directed graphs. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based. In order to thoroughly test the developed theory, we conduct experiments in real world settings, showcasing that directed spectral convolutional networks provide new state of the art results for heterophilic node classification on many datasets and -- as opposed to baselines -- may be rendered stable to resolution-scale varying topological perturbations.

Motivation & Objective

• To overcome the longstanding limitation that spectral graph networks require undirected graphs due to the need for a well-defined graph Fourier transform.
• To develop a theoretically sound framework for spectral convolutions on directed graphs without relying on self-adjoint operators or eigendecompositions.
• To investigate the role of basis choice and characteristic operators in filter design for directed graphs.
• To empirically validate the new framework on real-world datasets, particularly in scenarios with topological perturbations.
• To demonstrate improved generalization and stability in heterophilic node classification tasks on directed graphs.

Proposed method

• Replaces the traditional graph Fourier transform with a resolvent-based spectral framework using the resolvent operator $ R_z(L) = (L - zI)^{-1} $ for complex $ z $.
• Defines spectral filters as functions of the resolvent, enabling frequency-response interpretations even on non-self-adjoint operators.
• Uses complex analysis and spectral theory to ensure mathematical consistency and stability of filters on directed graphs.
• Employs a basis of generalized eigenvectors or other spectral decompositions to express filters, analyzing their impact on model performance.
• Introduces a graph-level aggregation scheme $ \Omega $ to combine node features into graph-level representations.
• Applies the framework to real-world molecular graphs (QM7 dataset), using atomic charges and Coulomb weights to define node and edge weights.

Experimental results

Research questions

• RQ1 Can spectral convolutions be meaningfully extended to directed graphs without relying on the graph Fourier transform?
• RQ2 How do different choices of basis and characteristic operators affect the expressivity and stability of spectral filters on directed graphs?
• RQ3 What is the impact of topological perturbations on model performance, and can the proposed framework maintain robustness?
• RQ4 Can the new framework achieve state-of-the-art performance in heterophilic node classification on directed graphs?
• RQ5 How does the resolvent-based approach compare to existing spectral and spatial methods in terms of stability and generalization?

Key findings

• HoloNets achieve new state-of-the-art performance in heterophilic node classification across multiple benchmark datasets, including QM7.
• The model demonstrates stability to resolution-scale topological perturbations, such as deflections of hydrogen atoms in molecular graphs, unlike existing baselines.
• The framework successfully generalizes to unseen graphs, as evidenced by consistent performance across different random seeds in experiments.
• The use of resolvent-based filters enables a frequency-response interpretation even on non-self-adjoint operators, validating the theoretical foundation.
• Aggregation via $ \Omega $ and node-weighting by atomic charges lead to effective graph-level representations, particularly in coarse-grained settings.
• The choice of $ z = -1 $ in the resolvent $ R_{-1}(\Delta) = (\Delta + I)^{-1} $ yields stable and effective filtering behavior in practice.