[Paper Review] Combining Neural Networks with Personalized PageRank for Classification on Graphs

Johannes Klicpera, Aleksandar Bojchevski+1 more|arXiv (Cornell University)|Oct 14, 2018

Advanced Graph Neural Networks52 citations

TL;DR

This paper proposes PPNP and APPNP, models that combine neural networks with personalized PageRank to improve graph classification by leveraging large, adjustable neighborhoods. By replacing standard message passing with PageRank-based propagation, the method achieves state-of-the-art performance with comparable or lower training time and parameters, outperforming recent GCN-like models in a comprehensive evaluation.

ABSTRACT

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models. Our implementation is available online.

Motivation & Objective

• To address the limited receptive field of existing graph neural networks that only consider nodes within a few propagation steps.
• To improve classification performance on graph-structured data by expanding the neighborhood used for message passing.
• To develop a method that maintains low training time and parameter count while enabling flexible, large-scale neighborhood aggregation.
• To create a framework that is compatible with any neural network architecture for graph classification.
• To provide a thorough empirical evaluation comparing the proposed method against recent GCN-based models.

Proposed method

• The method derives a connection between graph convolutional networks (GCN) and personalized PageRank to design a new propagation scheme.
• It introduces personalized propagation of neural predictions (PPNP), which uses the full personalized PageRank matrix to propagate predictions across the graph.
• A fast approximation, APPNP, is proposed by replacing the full PageRank matrix with a fixed-point iteration, reducing computational cost.
• The propagation step is integrated into a neural network framework, allowing end-to-end training with any neural network architecture.
• The method enables large, adjustable receptive fields by leveraging the stationary distribution of personalized PageRank.
• The model is trained using standard backpropagation, with the propagation step differentiable via the PageRank formulation.

Experimental results

Research questions

• RQ1 Can a graph neural network model achieve better performance by using a larger, more flexible neighborhood than standard GCN message passing?
• RQ2 Does replacing standard message passing with personalized PageRank propagation lead to improved generalization on semi-supervised node classification tasks?
• RQ3 Can the proposed method maintain low training time and parameter count while outperforming state-of-the-art GCN-like models?
• RQ4 How does the model scale with respect to neighborhood size and graph size in comparison to existing methods?
• RQ5 Is the method robust and effective across diverse graph datasets with varying structures and sparsity?

Key findings

• PPNP and APPNP achieve state-of-the-art performance on semi-supervised node classification across multiple benchmark datasets.
• The models outperform several recently proposed GCN-like models in the most comprehensive evaluation to date.
• Training time for PPNP and APPNP is on par or faster than previous models, despite using larger neighborhoods.
• The number of parameters in PPNP and APPNP is on par or lower than competing models, ensuring efficiency.
• APPNP achieves performance close to PPNP while being significantly faster, making it suitable for large-scale graphs.
• The method demonstrates strong generalization by effectively utilizing long-range dependencies through personalized PageRank.