[Paper Review] Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Johannes Gasteiger, Aleksandar Bojchevski+1 more|arXiv (Cornell University)|Oct 14, 2018

Advanced Graph Neural Networks431 citations

TL;DR

The paper introduces PPNP and its fast variant APPNP, decoupling prediction from propagation by using personalized PageRank to propagate neural predictions over large neighborhoods, achieving state-of-the-art results with few parameters and efficient training.

ABSTRACT

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models. Our implementation is available online.

Motivation & Objective

• Motivate the limitation of classic GCNs in using only small neighborhoods due to oversmoothing.
• Propose a propagation scheme based on personalized PageRank to preserve locality while leveraging large neighborhoods.
• Decouple the prediction network from the propagation mechanism for scalability and flexibility.
• Provide an end-to-end trainable model with competitive accuracy and efficiency across multiple graph datasets.

Proposed method

• Relate Graph Convolutional Networks to PageRank and extend to personalized PageRank with a root-node teleport vector.
• Define PPNP where node predictions from a neural network are propagated via a personalized PageRank scheme: Z_PPNP = softmax(alpha(I - (1-alpha)A_hat)^-1 H).
• Introduce APPNP as a scalable, approximate version using power iteration: Z^(0)=H; Z^(k+1)=(1-alpha)A_hat Z^(k) + alpha H; Z^(K)=softmax(...).
• Use teleport probability alpha to control locality versus global influence and enable very deep effective propagation without oversmoothing.
• Train end-to-end, with H produced by a neural network f_theta(X).
• Maintain sparsity by avoiding dense matrix inversions and making propagation independent of the predictor network.

Experimental results

Research questions

• RQ1 Can a PageRank-based propagation scheme preserve locality while using a large neighborhood for node classification?
• RQ2 Does decoupling prediction from propagation improve efficiency and scalability of GNNs without sacrificing accuracy?
• RQ3 How does the teleport parameter alpha and the number of propagation steps K affect performance across datasets?
• RQ4 Can APPNP approximate PPNP accurately with linear complexity while preserving or improving accuracy?
• RQ5 Is propagation beneficial during training, inference, or both for pretrained networks?

Key findings

• PPNP and APPNP consistently outperform several state-of-the-art GCN-like models on multiple graph datasets under a rigorous evaluation protocol.
• APPNP achieves near-exact PPNP performance with linear time complexity and preserves graph sparsity.
• Propagation with personalized PageRank improves accuracy, especially in settings with few labeled nodes and larger effective neighborhoods.
• The models maintain competitive training times, with APPNP generally faster than more complex alternatives and scalable to larger graphs.
• A thorough statistical evaluation (100 random splits, bootstrapped CIs, paired t-tests) supports the reported gains over baselines.
• Propagating during inference significantly improves accuracy even when the predictor is trained without graph information.