[Paper Review] Scalable Graph Neural Networks via Bidirectional Propagation

Ming Chen, Zhewei Wei+5 more|arXiv (Cornell University)|Oct 29, 2020

Advanced Graph Neural NetworksComputer Science45 references46 citations

TL;DR

GBP introduces a scalable GNN with localized bidirectional propagation that yields a sub-linear time complexity for precomputation and training, enabling effective learning on billion-edge graphs on a single machine.

ABSTRACT

Graph Neural Networks (GNN) is an emerging field for learning on non-Euclidean data. Recently, there has been increased interest in designing GNN that scales to large graphs. Most existing methods use "graph sampling" or "layer-wise sampling" techniques to reduce training time. However, these methods still suffer from degrading performance and scalability problems when applying to graphs with billions of edges. This paper presents GBP, a scalable GNN that utilizes a localized bidirectional propagation process from both the feature vectors and the training/testing nodes. Theoretical analysis shows that GBP is the first method that achieves sub-linear time complexity for both the precomputation and the training phases. An extensive empirical study demonstrates that GBP achieves state-of-the-art performance with significantly less training/testing time. Most notably, GBP can deliver superior performance on a graph with over 60 million nodes and 1.8 billion edges in less than half an hour on a single machine. The codes of GBP can be found at https://github.com/chennnM/GBP .

Motivation & Objective

• Motivate scalable GNNs that can handle graphs with billions of edges.
• Develop a propagation framework that decouples feature propagation from neural network learning.
• Achieve sub-linear time complexity for precomputation and training while preserving accuracy.
• Provide empirical evidence across small to billion-scale graphs showing performance and efficiency gains.

Proposed method

• Define a Generalized PageRank propagation matrix P that combines multi-hop diffusion with feature propagation.
• Compute P via a bidirectional propagation approach combining Monte-Carlo from training nodes and deterministic Reverse Push from features.
• Use P as a fixed propagation matrix, enabling standard mini-batch training of a neural network on top of P.
• Show that with appropriate settings, the estimator for P is unbiased and sub-linear in time.
• Demonstrate that GBP can emulate various graph convolutions by adjusting the weight sequence w_ell and the diffusion parameter r.
• Propose a practical, parallelizable algorithm (Algorithm 1) that achieves the bidirectional propagation efficiently.

Experimental results

Research questions

• RQ1 Can a localized bidirectional propagation framework yield sub-linear time complexity for precomputation and training in GNNs?
• RQ2 How does decoupling feature propagation from neural network learning affect semi-supervised and supervised tasks on large-scale graphs?
• RQ3 To what extent can GBP match or exceed state-of-the-art scalable GNNs on various benchmarks while reducing computation time?
• RQ4 Is GBP capable of scaling to billion-edge graphs on a single machine without sacrificing predictive performance?

Key findings

• GBP achieves sub-linear time complexity for precomputation and training in theory.
• GBP delivers state-of-the-art or competitive performance with significantly reduced training/testing time on multiple datasets.
• On a graph with over 60 million nodes and 1.8 billion edges, GBP runs in less than half an hour on a single machine.
• For inductive learning on large graphs, GBP attains comparable accuracy to GraphSAINT with 5–10x less running time.
• GBP demonstrates strong performance even with random features on billion-scale Friendster, highlighting its ability to capture graph structure.