[Paper Review] Semi-Supervised Classification with Graph Convolutional Networks

Thomas Kipf, Max Welling|arXiv (Cornell University)|Sep 9, 2016

Advanced Graph Neural Networks22 references8,058 citations

TL;DR

Introduces a scalable Graph Convolutional Network (GCN) for semi-supervised node classification that directly operates on graphs, using a renormalized feature propagation layer and achieving strong results on citation networks and a knowledge graph dataset.

ABSTRACT

We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.

Motivation & Objective

• Motivate graph-structured semi-supervised learning where labels are scarce.
• Propose a simple, scalable graph-based neural network layer that directly uses graph structure and node features.
• Demonstrate improved accuracy and efficiency over state-of-the-art baselines on multiple datasets.

Proposed method

• Define a layer-wise propagation rule H^{(l+1)} = σ( H^{(l)} W^{(l)}) with a renormalization trick  = âA with self-loops and degree-based normalization.
• Show that a first-order approximation of spectral graph convolutions yields a practical, efficient GCN (Eq. 7).
• Use a two-layer GCN with softmax for semi-supervised node classification on labeled nodes only, optimizing cross-entropy loss (Eq. 10).
• Compute Â̂ = D̃^{−1/2} Ẫ D̃^{−1/2} to keep training stable and scalable to large graphs.
• Provide a GPU-accelerated TensorFlow implementation with linear complexity in the number of edges (O(|E|F C)).
• Evaluate with fixed datasets and compare to LP, ManiReg, SemiEmb, DeepWalk, ICA, and Planetoid baselines.

Experimental results

Research questions

• RQ1 Can a simple, renormalized graph convolutional layer effectively propagate information across a graph for semi-supervised node classification?
• RQ2 Does incorporating adjacency structure directly into a neural network improve accuracy and efficiency over graph-Laplacian regularization and embedding-based methods?
• RQ3 What is the impact of the propagation design (renormalization trick) on performance and scalability across different datasets?
• RQ4 How does the GCN perform on large-scale knowledge-graph derived data (NELL) compared to citation networks?

Key findings

• GCN achieves higher accuracy than several baselines on Citeseer (70.3%), Cora (81.5%), Pubmed (79.0%), and NELL (66.0%).
• The renormalization trick improves efficiency and predictive performance over higher-order Chebyshev or first-order models.
• Full-batch training with sparse adjacency yields linear time complexity in the number of edges, suitable for large graphs.
• Deeper variants (up to 10 layers) are explored, with the two-layer model performing strongly across datasets.
• On random graphs, the model demonstrates scalable training times on GPU and CPU implementations.