[Paper Review] Simplifying Graph Convolutional Networks

Felix Wu, Tianyi Zhang+4 more|arXiv (Cornell University)|Feb 19, 2019

Advanced Graph Neural Networks57 references1,182 citations

TL;DR

This paper introduces Simple Graph Convolution (SGC), a linearized variant of GCN that precomputes a fixed graph-based feature smoothing and trains a simple logistic regression, achieving competitive accuracy with much faster training on large graphs.

ABSTRACT

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

Motivation & Objective

• Motivate simplification of GCNs by removing unnecessary nonlinearities and layer-wise transformations.
• Derive and analyze a linear model that preserves the receptive field of multi-layer GCNs.
• Demonstrate that a fixed graph-based feature smoothing followed by logistic regression can match or exceed GCN performance.
• Show that precomputing feature propagation enables significant computational efficiency on large graphs.
• Evaluate SGC across citation networks, social networks, and diverse downstream tasks to establish wide applicability.

Proposed method

• Define the graph G with adjacency A, degree D, and feature matrix X.

Experimental results

Research questions

• RQ1 Does removing nonlinearities between GCN layers and collapsing weights yield a competitive model?
• RQ2 How does fixed feature propagation H = S^K X influence performance and interpretability?
• RQ3 Can SGC scale to large graphs while maintaining accuracy across tasks?
• RQ4 What theoretical interpretation can explain SGC’s effectiveness from a spectral perspective?

Key findings

• SGC achieves competitive test accuracy compared to GCN and state-of-the-art graph networks on citation networks (Cora, Citeseer, Pubmed) and Reddit.
• SGC is significantly faster, with precomputation of S^K X enabling training up to two orders of magnitude faster on Reddit.
• SGC maintains strong performance on downstream tasks such as text classification, user geolocation, relation extraction, and zero-shot image classification.
• Theoretical analysis shows SGC corresponds to a fixed low-pass filter on the graph spectrum, and the renormalization trick shrinks the spectrum yielding smoothing.
• SGC often uses fewer parameters and avoids overfitting, sometimes outperforming GCN-based models due to reduced complexity.