[Paper Review] $p$-Laplacian Based Graph Neural Networks

Guoji Fu, Peilin Zhao+1 more|arXiv (Cornell University)|Nov 14, 2021

Advanced Graph Neural Networks72 references4 citations

TL;DR

This paper proposes p GNN, a graph neural network that uses a p-Laplacian-based message passing mechanism derived from a discrete regularization framework, enabling effective learning on both homophilic and heterophilic graphs. The method acts as a low-high-pass filter in the spectral domain, adaptively learning aggregation weights and outperforming state-of-the-art GNNs on heterophilic benchmarks while remaining robust to noisy edges.

ABSTRACT

Graph neural networks (GNNs) have demonstrated superior performance for semi-supervised node classification on graphs, as a result of their ability to exploit node features and topological information simultaneously. However, most GNNs implicitly assume that the labels of nodes and their neighbors in a graph are the same or consistent, which does not hold in heterophilic graphs, where the labels of linked nodes are likely to differ. Hence, when the topology is non-informative for label prediction, ordinary GNNs may work significantly worse than simply applying multi-layer perceptrons (MLPs) on each node. To tackle the above problem, we propose a new $p$-Laplacian based GNN model, termed as $^p$GNN, whose message passing mechanism is derived from a discrete regularization framework and could be theoretically explained as an approximation of a polynomial graph filter defined on the spectral domain of $p$-Laplacians. The spectral analysis shows that the new message passing mechanism works simultaneously as low-pass and high-pass filters, thus making $^p$GNNs are effective on both homophilic and heterophilic graphs. Empirical studies on real-world and synthetic datasets validate our findings and demonstrate that $^p$GNNs significantly outperform several state-of-the-art GNN architectures on heterophilic benchmarks while achieving competitive performance on homophilic benchmarks. Moreover, $^p$GNNs can adaptively learn aggregation weights and are robust to noisy edges.

Motivation & Objective

• To address the limitation of existing GNNs in heterophilic graphs, where connected nodes often have different labels.
• To improve GNN performance on graphs with non-informative or noisy topologies, where standard GNNs degrade below MLP performance.
• To develop a theoretically grounded message passing mechanism that generalizes beyond the standard 2-Laplacian and adapts to local node embedding variations.
• To enable GNNs to work effectively across both homophilic and heterophilic graph settings without prior knowledge of graph homophily.

Proposed method

• Proposes a p-Laplacian based message passing mechanism derived from a discrete regularization framework, approximating a polynomial graph filter on the spectral domain of the p-Laplacian.
• The message passing rule is theoretically justified as an approximation of a graph filter that acts as a low-high-pass filter, depending on the value of p.
• Introduces adaptive aggregation weights that depend on the local variation of node embeddings, measured by the graph gradient norm, enabling dynamic filtering behavior.
• Designs a new GNN architecture, p GNN, which generalizes existing models like SGC, APPNP, and GPRGNN when p=2.
• Employs spectral analysis to show that the p-Laplacian filter can emphasize both low and high-frequency components in node representations.
• Uses a learnable parameter p to control the filter behavior, allowing the model to self-adapt to the graph's homophily level and noise characteristics.

Experimental results

Research questions

• RQ1 Can a p-Laplacian-based message passing mechanism improve GNN performance on heterophilic graphs where standard GNNs fail?
• RQ2 How does the p-Laplacian message passing mechanism behave spectrally, and can it act as a low-high-pass filter to preserve both smooth and oscillatory patterns?
• RQ3 Does the proposed p GNN model adaptively learn aggregation weights based on local node embedding variation, especially in noisy or non-informative topologies?
• RQ4 Can p GNN achieve competitive performance on homophilic graphs while significantly outperforming baselines on heterophilic benchmarks?
• RQ5 How robust is p GNN to edge noise, and does it maintain performance even when topology is uninformative?

Key findings

• p GNN achieves state-of-the-art performance on heterophilic benchmarks, outperforming GCN, GAT, and GPRGNN on datasets like Chameleon and Squirrel.
• On the Cora, CiteSeer, and PubMed homophilic datasets, p GNN achieves competitive results, with 2.0 GNN attaining 95.05% accuracy on Physics, matching or exceeding SGC and GCN.
• For the Chameleon dataset, 1.5 GNN achieves 86.94% accuracy, significantly outperforming GCN (84.12%) and GAT (86.50%).
• On synthetic cSBM datasets with varying levels of heterophily, p GNN maintains high performance across all settings, demonstrating robustness to graph structure shifts.
• In noisy edge experiments, p GNN performs as well as MLPs on completely random graphs, while standard GNNs degrade significantly.
• Visualization via t-SNE shows that p GNN learns more discriminative and well-separated node representations than GCN and other baselines on both homophilic and heterophilic graphs.