[Paper Review] SIGN: Scalable Inception Graph Neural Networks

Fabrizio Frasca, Emanuele Rossi+4 more|arXiv (Cornell University)|Apr 23, 2020

Advanced Graph Neural NetworksComputer Science71 references187 citations

TL;DR

SIGN is a sampling-free Graph Neural Network that uses multi-scale, precomputable diffusion operators in an inception-like module to scale to web-scale graphs while achieving competitive or state-of-the-art results.

ABSTRACT

Graph representation learning has recently been applied to a broad spectrum of problems ranging from computer graphics and chemistry to high energy physics and social media. The popularity of graph neural networks has sparked interest, both in academia and in industry, in developing methods that scale to very large graphs such as Facebook or Twitter social networks. In most of these approaches, the computational cost is alleviated by a sampling strategy retaining a subset of node neighbors or subgraphs at training time. In this paper we propose a new, efficient and scalable graph deep learning architecture which sidesteps the need for graph sampling by using graph convolutional filters of different size that are amenable to efficient precomputation, allowing extremely fast training and inference. Our architecture allows using different local graph operators (e.g. motif-induced adjacency matrices or Personalized Page Rank diffusion matrix) to best suit the task at hand. We conduct extensive experimental evaluation on various open benchmarks and show that our approach is competitive with other state-of-the-art architectures, while requiring a fraction of the training and inference time. Moreover, we obtain state-of-the-art results on ogbn-papers100M, the largest public graph dataset, with over 110 million nodes and 1.5 billion edges.

Motivation & Objective

• Motivate scalable graph representation learning for web-scale graphs without sampling.
• Propose a lightweight, expressive architecture that precomputes diffusion operators to enable fast training and inference.
• Demonstrate competitive or state-of-the-art performance on large inductive and transductive node classification benchmarks.

Proposed method

• Introduce SIGN, a model with an inception-style module over multiple diffusion operators: Y = σ([XΘ0, A1XΘ1, ..., ArXΘr]); Z = concatenation of processed features; Y = ξ(ZΩ).
• Use precomputable diffusion operators Ai (powered variants of simple adjacency, Personalized PageRank, and triangle-based adjacency) to decouple computation from graph structure.
• Allow operators to be combined as SIGN(p,s,t) where p,s,t control the power counts of simple, PPR, and triangle-based adjacency, enabling flexible, task-driven receptive fields.
• Treat SIGN as a shallow architecture where graph filtering is computed once (precomputation) and subsequent MLP-style processing scales with N and d rather than graph size.
• Show that SIGN can emulate common GCN/S-GCN variants by choosing operators and activations appropriately (Table 2).
• Evaluate with Bayesian optimization and standard training tricks (Adam, dropout, batch norm, early stopping) on large-scale benchmarks.

Experimental results

Research questions

• RQ1 Can a sampling-free GNN with multi-operator, inception-style diffusion achieve competitive performance on large graphs without neighbor sampling?
• RQ2 How do different diffusion operators (simple adjacency, PPR, triangle-based, and their powers) affect performance across inductive and transductive tasks?
• RQ3 What is the trade-off between scalability (training/inference speed) and accuracy when using SIGN on web-scale datasets?
• RQ4 Can SIGN reach state-of-the-art results on the largest public graph benchmarks (e.g., ogbn-papers100M) while maintaining efficiency?
• RQ5 How does SIGN compare to sampling-based and other scalable GNNs in terms of convergence and runtime?

Key findings

• SIGN achieves state-of-the-art results on ogbn-papers100M among sampling-free methods, outperforming others by at least 1.8% in the transductive setting.
• On ogbn-papers100M, SIGN completes evaluation quickly, with one evaluation pass for validation in ~1.99s and test in ~3.34s (10 runs).
• SIGN outperforms sampling-free baselines on ogbn-products by at least 2.7% and attains competitive results on inductive datasets (Reddit, Flickr, PPI).
• SIGN shows substantial speedups over ClusterGCN and GraphSAINT in training, and is the fastest in inference across large graphs (up to 12 million nodes) by at least an order of magnitude.
• SIGN’s performance is robust to operator choices; best configurations are dataset-dependent, with PPR, triangle, and simple adjacency powers contributing differently across tasks.
• Precomputation of A1X, ..., ArX enables complexity comparable to an MLP, scaling as O(r Lff N d^2) and making training/inference independent of graph structure.