[Paper Review] GraphPatcher: Mitigating Degree Bias for Graph Neural Networks via Test-time Augmentation

Mingxuan Ju, Tong Zhao+3 more|arXiv (Cornell University)|Oct 1, 2023

Advanced Graph Neural Networks4 citations

TL;DR

GraphPatcher is a test-time augmentation framework that mitigates degree bias in graph neural networks by iteratively generating virtual nodes to patch corrupted ego-graphs of low-degree nodes, improving their performance without degrading high-degree node accuracy. It enhances GNNs on low-degree nodes by up to 6.5% and overall performance by up to 3.6%, outperforming state-of-the-art baselines while remaining model-agnostic and plug-and-play.

ABSTRACT

Recent studies have shown that graph neural networks (GNNs) exhibit strong biases towards the node degree: they usually perform satisfactorily on high-degree nodes with rich neighbor information but struggle with low-degree nodes. Existing works tackle this problem by deriving either designated GNN architectures or training strategies specifically for low-degree nodes. Though effective, these approaches unintentionally create an artificial out-of-distribution scenario, where models mainly or even only observe low-degree nodes during the training, leading to a downgraded performance for high-degree nodes that GNNs originally perform well at. In light of this, we propose a test-time augmentation framework, namely GraphPatcher, to enhance test-time generalization of any GNNs on low-degree nodes. Specifically, GraphPatcher iteratively generates virtual nodes to patch artificially created low-degree nodes via corruptions, aiming at progressively reconstructing target GNN's predictions over a sequence of increasingly corrupted nodes. Through this scheme, GraphPatcher not only learns how to enhance low-degree nodes (when the neighborhoods are heavily corrupted) but also preserves the original superior performance of GNNs on high-degree nodes (when lightly corrupted). Additionally, GraphPatcher is model-agnostic and can also mitigate the degree bias for either self-supervised or supervised GNNs. Comprehensive experiments are conducted over seven benchmark datasets and GraphPatcher consistently enhances common GNNs' overall performance by up to 3.6% and low-degree performance by up to 6.5%, significantly outperforming state-of-the-art baselines. The source code is publicly available at https://github.com/jumxglhf/GraphPatcher.

Motivation & Objective

• To address the persistent performance gap in GNNs between low-degree and high-degree nodes, especially under real-world power-law degree distributions.
• To overcome the limitation of existing methods that degrade high-degree node performance by creating artificial out-of-distribution training scenarios.
• To develop a model-agnostic, plug-and-play framework that enhances GNNs at test time without retraining or architectural changes.
• To maintain or improve GNN performance on high-degree nodes while significantly boosting performance on low-degree nodes through iterative virtual node generation.
• To enable effective degree bias mitigation for both supervised and self-supervised GNNs in practical deployment settings.

Proposed method

• GraphPatcher generates a sequence of ego-graphs with increasing corruption strength to simulate progressively degraded neighborhood information.
• It iteratively generates virtual nodes to patch the most corrupted ego-graphs, aiming to align the frozen GNN’s predictions on the patched graph with those on the corrupted version.
• The virtual nodes are optimized via a differentiable objective that minimizes the prediction discrepancy between the corrupted and patched ego-graphs.
• The framework is applied at test time, requiring no updates to the target GNN, making it model-agnostic and deployable as a plug-in module.
• It uses a multi-stage optimization process with multiple sampled ego-graphs per corruption level to stabilize training and improve generalization.
• The method is inspired by iterative diffusion processes but is specifically tailored to improve downstream GNN performance rather than data fidelity.

Figure 1 : The classification accuracy of GCN and SoTA frameworks that mitigate degree biases.

Experimental results

Research questions

• RQ1 Can a test-time augmentation framework improve GNN performance on low-degree nodes without degrading performance on high-degree nodes?
• RQ2 How can virtual node generation be optimized to preserve the original GNN’s inductive bias while enhancing predictions on sparse neighborhoods?
• RQ3 To what extent can a model-agnostic, plug-and-play framework mitigate degree bias across diverse graph datasets and GNN architectures?
• RQ4 Does the proposed method generalize to both supervised and self-supervised GNNs, including state-of-the-art models like GRAND?
• RQ5 How does the iterative virtual node generation strategy compare to existing data augmentation or graph generation techniques in terms of performance and efficiency?

Key findings

• GraphPatcher improves GNN performance on low-degree nodes by up to 6.5% across seven benchmark datasets, significantly outperforming existing baselines.
• It enhances overall GNN performance by up to 3.6%, demonstrating consistent gains across diverse graph types and architectures.
• The framework maintains or even improves the original GNN’s performance on high-degree nodes, avoiding the performance trade-off seen in prior methods.
• GraphPatcher is effective for both supervised and self-supervised GNNs, including state-of-the-art models like GRAND, where it further boosts performance beyond existing SoTA.
• The method is model-agnostic and requires no retraining or architectural changes, enabling seamless integration into existing production pipelines.
• The additional computational cost is manageable, as all ego-graphs are pre-generated to avoid redundant computation during optimization.

Figure 2 : GraphPatcher is presented ego-graphs corrupted by increasing strengths (i.e., the top half of the figure). From the most corrupted graph, it iteratively generates patching nodes to the anchor node, such that the target GNN behaves similarly given the currently patched graph or the corrupt