[论文解读] GraphPatcher: Mitigating Degree Bias for Graph Neural Networks via Test-time Augmentation

Mingxuan Ju, Tong Zhao+3 more|arXiv (Cornell University)|Oct 1, 2023

Advanced Graph Neural Networks被引用 4

一句话总结

GraphPatcher 是一种测试时增强框架，通过迭代生成虚拟节点来修补低度节点的受损自中心图，从而缓解图神经网络中的度偏差问题，提升其性能而不降低高阶节点的准确率。该方法在不降低高阶节点性能的前提下，将 GNN 在低度节点上的性能提升最高达 6.5%，整体性能提升最高达 3.6%，优于当前最先进基线方法，且保持模型无关性和即插即用特性。

ABSTRACT

Recent studies have shown that graph neural networks (GNNs) exhibit strong biases towards the node degree: they usually perform satisfactorily on high-degree nodes with rich neighbor information but struggle with low-degree nodes. Existing works tackle this problem by deriving either designated GNN architectures or training strategies specifically for low-degree nodes. Though effective, these approaches unintentionally create an artificial out-of-distribution scenario, where models mainly or even only observe low-degree nodes during the training, leading to a downgraded performance for high-degree nodes that GNNs originally perform well at. In light of this, we propose a test-time augmentation framework, namely GraphPatcher, to enhance test-time generalization of any GNNs on low-degree nodes. Specifically, GraphPatcher iteratively generates virtual nodes to patch artificially created low-degree nodes via corruptions, aiming at progressively reconstructing target GNN's predictions over a sequence of increasingly corrupted nodes. Through this scheme, GraphPatcher not only learns how to enhance low-degree nodes (when the neighborhoods are heavily corrupted) but also preserves the original superior performance of GNNs on high-degree nodes (when lightly corrupted). Additionally, GraphPatcher is model-agnostic and can also mitigate the degree bias for either self-supervised or supervised GNNs. Comprehensive experiments are conducted over seven benchmark datasets and GraphPatcher consistently enhances common GNNs' overall performance by up to 3.6% and low-degree performance by up to 6.5%, significantly outperforming state-of-the-art baselines. The source code is publicly available at https://github.com/jumxglhf/GraphPatcher.

研究动机与目标

• 为解决在真实世界幂律度分布下，GNN 在低度节点与高阶节点之间持续存在的性能差距。
• 克服现有方法因创建分布外训练场景而导致高阶节点性能下降的局限性。
• 开发一种模型无关、即插即用的框架，在测试时增强 GNN，无需微调或架构修改。
• 在显著提升低度节点性能的同时，保持或改善 GNN 在高阶节点上的性能，通过迭代生成虚拟节点实现。
• 实现在实际部署环境中，对监督学习与自监督学习 GNN 的有效度偏差缓解。

提出的方法

• GraphPatcher 生成一系列腐蚀强度逐步增强的自中心图，以模拟逐步退化的邻域信息。
• 它迭代地为最严重的自中心图生成虚拟节点，目标是使冻结的 GNN 在修补图上的预测结果与在腐蚀图上的预测结果对齐。
• 通过可微目标函数优化虚拟节点，以最小化腐蚀图与修补图之间预测结果的差异。
• 该框架在测试时应用，无需更新目标 GNN，因此具有模型无关性，可作为插件模块直接部署。
• 采用多阶段优化流程，每个腐蚀等级采样多个自中心图，以稳定训练并提升泛化能力。
• 该方法受迭代扩散过程启发，但专为提升下游 GNN 性能而设计，而非保持数据保真度。

Figure 1 : The classification accuracy of GCN and SoTA frameworks that mitigate degree biases.

实验结果

研究问题

• RQ1 测试时增强框架是否能在不降低高阶节点性能的前提下，提升 GNN 在低度节点上的表现？
• RQ2 如何优化虚拟节点生成过程，以在增强稀疏邻域预测能力的同时，保留原始 GNN 的归纳偏置？
• RQ3 一种模型无关、即插即用的框架在多大程度上能缓解多种图数据集与 GNN 架构中的度偏差？
• RQ4 所提出的方法是否能泛化至监督学习与自监督学习 GNN，包括最先进的模型如 GRAND？
• RQ5 与现有数据增强或图生成技术相比，该迭代虚拟节点生成策略在性能与效率方面表现如何？

主要发现

• GraphPatcher 在七个基准数据集上将 GNN 在低度节点上的性能提升最高达 6.5%，显著优于现有基线方法。
• 其整体 GNN 性能提升最高达 3.6%，在多种图类型与架构中均表现出一致的增益。
• 该框架在保持或甚至提升原始 GNN 在高阶节点上性能的同时，避免了以往方法中常见的性能权衡。
• GraphPatcher 对监督学习与自监督学习 GNN 均有效，包括最先进的模型如 GRAND，在后者中进一步提升了性能，超越现有最先进水平。
• 该方法具有模型无关性，无需微调或架构修改，可无缝集成至现有生产流水线。
• 额外计算开销可控，因所有自中心图均预先生成，避免了优化过程中的重复计算。

Figure 2 : GraphPatcher is presented ego-graphs corrupted by increasing strengths (i.e., the top half of the figure). From the most corrupted graph, it iteratively generates patching nodes to the anchor node, such that the target GNN behaves similarly given the currently patched graph or the corrupt