[Paper Review] Personalized Embedding Propagation: Combining Neural Networks on Graphs with Personalized PageRank.

Johannes Klicpera, Aleksandar Bojchevski+1 more|arXiv (Cornell University)|Oct 14, 2018

Advanced Graph Neural Networks21 citations

TL;DR

This paper proposes Personalized Embedding Propagation (PEP) and its approximation PEP_A, which combine graph neural networks with personalized PageRank to enable large, adjustable neighborhoods for node classification. By leveraging the mathematical link between GCNs and PageRank, PEP achieves state-of-the-art performance on semi-supervised graph classification with comparable or faster training and fewer parameters than prior methods.

ABSTRACT

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood cannot be easily extended. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct personalized embedding propagation (PEP) and its approximation, PEP$_ ext{A}$. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification on multiple graphs in the most thorough study done so far for GCN-like models.

Motivation & Objective

• To address the limitation of existing graph neural networks that only consider nodes within a few propagation steps.
• To develop a scalable and flexible neighborhood aggregation mechanism that can be adjusted to include distant nodes in the graph.
• To maintain or improve model performance while reducing training time and parameter count compared to existing GCN-based models.
• To create a general-purpose propagation framework compatible with any neural network architecture on graphs.
• To empirically validate the method's superiority across diverse graph datasets in a comprehensive benchmarking study.

Proposed method

• Derives a theoretical connection between graph convolutional networks (GCN) and personalized PageRank to inform a new message-passing mechanism.
• Designs a personalized embedding propagation (PEP) scheme that uses personalized PageRank vectors as aggregation weights for node features.
• Introduces PEP_A, an efficient approximation of PEP that reduces computational cost while preserving performance.
• Employs a learnable personalization parameter in PageRank to control the effective neighborhood size dynamically.
• Integrates the PEP mechanism as a differentiable layer that can be combined with any neural network architecture on graphs.
• Uses the closed-form solution of personalized PageRank to enable fast and scalable computation over large graphs.

Experimental results

Research questions

• RQ1 Can a personalized PageRank-based aggregation mechanism improve the receptive field of GCN-like models beyond local neighborhoods?
• RQ2 Does the proposed PEP method achieve better classification accuracy than existing GCN-based models on semi-supervised graph learning benchmarks?
• RQ3 Can PEP maintain or reduce training time and parameter count while expanding the effective neighborhood size?
• RQ4 How does the performance of PEP scale across different graph datasets and architectures?
• RQ5 To what extent can PEP be combined with various neural network architectures without architectural modifications?

Key findings

• PEP outperforms several recently proposed GCN-like models on multiple benchmark graphs in the most comprehensive evaluation to date.
• The model achieves state-of-the-art performance on semi-supervised node classification tasks with a larger, adjustable receptive field.
• PEP maintains training time comparable to or faster than existing models, despite using a broader neighborhood.
• The number of parameters in PEP is on par with or lower than those of competing models, indicating efficiency.
• The approximation PEP_A preserves most of the performance of PEP while significantly reducing computational overhead.
• The method is general and can be seamlessly integrated with any neural network architecture on graph-structured data.