[Paper Review] N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification

Sami Abu-El-Haija, Amol Kapoor+2 more|arXiv (Cornell University)|Feb 24, 2018

Advanced Graph Neural Networks19 references78 citations

TL;DR

N-GCN trains multiple GCN instances on different powers of the normalized adjacency matrix to capture multi-scale graph information, and learns a joint classifier that combines their outputs to improve semi-supervised node classification. It achieves state-of-the-art results on several benchmarks and extends to N-SAGE variants.

ABSTRACT

Graph Convolutional Networks (GCNs) have shown significant improvements in semi-supervised learning on graph-structured data. Concurrently, unsupervised learning of graph embeddings has benefited from the information contained in random walks. In this paper, we propose a model: Network of GCNs (N-GCN), which marries these two lines of work. At its core, N-GCN trains multiple instances of GCNs over node pairs discovered at different distances in random walks, and learns a combination of the instance outputs which optimizes the classification objective. Our experiments show that our proposed N-GCN model improves state-of-the-art baselines on all of the challenging node classification tasks we consider: Cora, Citeseer, Pubmed, and PPI. In addition, our proposed method has other desirable properties, including generalization to recently proposed semi-supervised learning methods such as GraphSAGE, allowing us to propose N-SAGE, and resilience to adversarial input perturbations.

Motivation & Objective

• Motivate combining GCNs with random-walk based multi-scale graph information for semi-supervised node classification.
• Develop a network of GCNs (N-GCN) that processes multiple powers of the normalized adjacency matrix.
• Jointly train all GCN modules and a classifier to select useful multi-scale representations.
• Demonstrate state-of-the-art performance on standard datasets and analyze robustness to perturbations.

Proposed method

• Create K instantiations of GCNs, each fed with a different power of the normalized adjacency matrix  A (including  A^0 = I).
• Concatenate outputs from the K GCNs and feed them to a fully-connected classifier (N-GCN_fc).
• Alternatively, learn a soft-attention combination over GCN outputs (N-GCN_a) to weight scales adaptively.
• Train end-to-end with cross-entropy loss on labeled nodes, optionally applying intermediate supervision to encourage each GCN to be useful.
• Generalize to other graph models (e.g., GraphSAGE) via a Network-of-Graph-Models framework.
• Evaluate on standard graph benchmarks (Cora, Citeseer, PubMed, PPI) and compare against baselines (GCN, SAGE, DCNN, etc.).

Experimental results

Research questions

• RQ1 Does explicitly leveraging multi-scale information from different random walk steps improve semi-supervised node classification beyond vanilla GCNs?
• RQ2 Can a network of GCNs operating on powers of the adjacency matrix learn to combine distant and local information effectively?
• RQ3 Is the proposed N-GCN robust to input perturbations and feature noise?
• RQ4 How does N-GCN compare to related methods (e.g., DCNN, GraphSAGE) on benchmark datasets?
• RQ5 Can the approach generalize to other graph models beyond GCNs (e.g., SAGE)?

Key findings

• N-GCN achieves state-of-the-art accuracy on Cora, Citeseer, PubMed, and PPI datasets compared to strong baselines.
• Using multi-scale GCNs (K>1) improves performance over the single-scale GCN, with larger gains when increasing K.
• Attention-based and FC-classification variants (N-GCN_a, N-GCN_fc) both outperform baselines, showing effectiveness of multi-scale fusion.
• Under feature perturbations, N-GCN and related multi-scale models outperform vanilla GCN and SAGE, indicating resilience to missing features.
• Deeper single-path GCN/SAGE does not consistently improve accuracy, whereas N-GCN benefits from multi-scale, shallow components.