[Paper Review] Adaptive Universal Generalized PageRank Graph Neural Network

Eli Chien, Jianhao Peng+2 more|arXiv (Cornell University)|Jun 14, 2020

Advanced Graph Neural NetworksComputer Science56 references93 citations

TL;DR

GPR-GNN adaptively learns Generalized PageRank weights to jointly exploit node features and graph topology, achieving strong performance across homophilic and heterophilic graphs and mitigating over-smoothing without sacrificing depth.

ABSTRACT

In many important graph data processing applications the acquired information includes both node features and observations of the graph topology. Graph neural networks (GNNs) are designed to exploit both sources of evidence but they do not optimally trade-off their utility and integrate them in a manner that is also universal. Here, universality refers to independence on homophily or heterophily graph assumptions. We address these issues by introducing a new Generalized PageRank (GPR) GNN architecture that adaptively learns the GPR weights so as to jointly optimize node feature and topological information extraction, regardless of the extent to which the node labels are homophilic or heterophilic. Learned GPR weights automatically adjust to the node label pattern, irrelevant on the type of initialization, and thereby guarantee excellent learning performance for label patterns that are usually hard to handle. Furthermore, they allow one to avoid feature over-smoothing, a process which renders feature information nondiscriminative, without requiring the network to be shallow. Our accompanying theoretical analysis of the GPR-GNN method is facilitated by novel synthetic benchmark datasets generated by the so-called contextual stochastic block model. We also compare the performance of our GNN architecture with that of several state-of-the-art GNNs on the problem of node-classification, using well-known benchmark homophilic and heterophilic datasets. The results demonstrate that GPR-GNN offers significant performance improvement compared to existing techniques on both synthetic and benchmark data.

Motivation & Objective

• Address the limitations of traditional GNNs that rely on a fixed bias toward homophily or heterophily.
• Develop a universal GNN architecture that adaptively fuses node features with graph topology via Generalized PageRank (GPR).
• Enable deep propagation without over-smoothing by learning propagation weights end-to-end.
• Provide theoretical insights linking GPR to polynomial graph filtering and demonstrate practical performance on synthetic and real datasets.

Proposed method

• Introduce the GPR-GNN architecture that first extracts hidden features per node via a neural network, then propagates these features using a Generalized PageRank (GPR) with learnable weights γ_k.
• Represent the propagation as H^(k) = Ã_sym H^(k-1) with H^(0) = f_θ(X), where γ_k weight the contributions across K propagation steps and Z = ∑_{k=0}^K γ_k H^(k).
• Learn the GPR weights γ_k jointly with network parameters θ in an end-to-end manner, allowing positive and negative weights to adapt to the homophily/heterophily of the graph.
• Interpret the model as a polynomial graph filter g_{γ,K}(Λ) with g_{γ,K}(λ) = ∑_{k=0}^K γ_k λ^k, enabling analysis of low-pass and high-pass behavior.
• Theoretical results show that nonnegative, summing γ_k produce low-pass filters, while allowing negative γ_k yields high-pass filtering suitable for heterophilic graphs, and that adaptive γ_k mitigates over-smoothing (Theorems 4.1 and 4.2).
• Empirically evaluate on synthetic contextual stochastic block model (cSBM) data and real-world datasets across homophilic and heterophilic regimes; compare with standard GNNs and PPR-based methods.

Experimental results

Research questions

• RQ1 Can a GNN that adaptively learns GPR weights achieve universal performance across graphs with varying levels of homophily and heterophily?
• RQ2 Do learned GPR weights provide interpretable insights into when feature propagation versus topological propagation is more informative?
• RQ3 Can adaptive GPR weights mitigate over-smoothing and allow deeper propagation without performance loss?
• RQ4 How does GPR-GNN perform on synthetic cSBM benchmarks and real-world homophilic/heterophilic datasets compared to state-of-the-art baselines?

Key findings

• GPR-GNN outperforms baselines on synthetic cSBM data across the homophily-heterophily spectrum, with especially strong gains in heterophilic settings.
• On real-world benchmarks, GPR-GNN achieves state-of-the-art results on homophilic datasets and significant improvements on heterophilic datasets compared to APPNP, SGC, GCN, GAT, JKNet, GCN-Cheby, and SAGE.
• Learned GPR weights are positive on homophilic datasets and exhibit negative/zig-zag patterns on heterophilic datasets, aligning with theoretical expectations about high-pass filtering.
• The model remains robust under random GPR initialization, particularly in dense splits, and shows interpretability through the learned γ_k patterns.
• GPR-GNN can escape over-smoothing by reducing the magnitude of high-step γ_k when necessary, enabling informative large-step propagation when beneficial.
• Compared to fixed-weights PPR-based methods, adaptive γ_k allow the graph filter to capture both low- and high-frequency components, addressing heterophily.