[Paper Review] Revisiting Graph Neural Networks: All We Have is Low-Pass Filters

Hoang Nt, Takanori Maehara|arXiv (Cornell University)|May 23, 2019

Advanced Graph Neural Networks32 references248 citations

TL;DR

The paper analyzes GNNs through graph signal processing and shows that GNNs mainly perform low-pass filtering; learning is often unnecessary, and gfNN serves as an effective fast baseline denoising approach. It provides theoretical and empirical evidence across benchmarks and introduces a low-pass graph filter neural network.

ABSTRACT

Graph neural networks have become one of the most important techniques to solve machine learning problems on graph-structured data. Recent work on vertex classification proposed deep and distributed learning models to achieve high performance and scalability. However, we find that the feature vectors of benchmark datasets are already quite informative for the classification task, and the graph structure only provides a means to denoise the data. In this paper, we develop a theoretical framework based on graph signal processing for analyzing graph neural networks. Our results indicate that graph neural networks only perform low-pass filtering on feature vectors and do not have the non-linear manifold learning property. We further investigate their resilience to feature noise and propose some insights on GCN-based graph neural network design.

Motivation & Objective

• Motivate a graph signal processing perspective to understand when GNNs work for vertex classification.
• Show that standard GCN/SGC operations primarily perform low-pass filtering and denoise features.
• Introduce gfNN as a fast, noise-tolerant baseline and compare it with GCN and SGC.
• Theoretically characterize the effect of graph-based low-pass filtering on true vs. noisy features.

Proposed method

• Model graph signals with graph Laplacians and generalized eigenvectors to define graph Fourier transform.
• Demonstrate that propagation with adjacency/normalized Laplacian matrices implements low-pass filtering.
• Prove that low-pass filtering approximates the true feature recovery and optimization solution (Theorems 2, 3, 7, 8).
• Propose gfNN (graph filter neural network) as a two-step pipeline: apply graph filters, then learn with a classifier.
• Provide a bias-variance analysis for low-pass filters under Assumption 1 (low-frequency true features with noise).
• Conduct experiments on real-world datasets (Cora, Citeseer, Pubmed, Reddit, PPI, Two Circles) and synthetic data to compare GCN, SGC, gfNN.

Experimental results

Research questions

• RQ1 Under Assumption 1 (low-frequency true features with noise), do GCN, SGC, and gfNN yield similar performance to models trained on true features?
• RQ2 Do graph-induced low-pass filters inherently denoise noisy features, and how does this affect learning and robustness to noise?
• RQ3 Can a simple gfNN baseline match or exceed traditional GCN/SGC performance while offering faster training and better noise tolerance?
• RQ4 In nonlinear feature spaces, can SGC fail where nonlinear gfNN/GCN succeed, illustrating limits of linear propagation?

Key findings

• Low-frequency components primarily carry information for classification; adding high-frequency components or noise can degrade performance.
• Multiplying by adjacency/augmented adjacency filters acts as a low-pass filter, shrinking all eigenvalues monotonically with self-loops.
• Under realistic assumptions, gfNN and GCN yield similar results to models trained on true features when true features are low-frequency.
• gfNN is faster (no training-time graph multiplications) and more robust to feature noise than GCN.
• SGC behaves like a linear model on true features and can fail on nonlinear separability, unlike gfNN/GCN in nonlinear settings.
• Experiments show gfNN and SGC achieve comparable accuracy to GCN and better noise tolerance across several datasets; GCN can overfit when noise is high.