[论文解读] Graph Convolutional Networks against Degree-Related Biases.

Xianfeng Tang, Huaxiu Yao+6 more|arXiv (Cornell University)|Jun 28, 2020

Advanced Graph Neural Networks被引用 4

一句话总结

该论文提出SL-DSGCN，一种自监督的、与度数相关的图卷积网络，通过引入度数感知的消息传递机制和不确定性感知的伪标签生成方法，减轻了高阶节点的准确率偏差。该方法在基准数据集上显著提升了低阶节点的性能，优于最先进的自训练和自监督GCN方法。

ABSTRACT

In recent years, Graph Convolutional Networks (GCNs) show competitive performance in different domains, such as social network analysis, recommendation, and smart city. However, training GCNs with insufficient supervision is very difficult. The performance of GCNs becomes unsatisfying with few labeled data. Although some pioneering work try to understand why GCNs work or fail, their analysis focus more on the entire model level. Profiling GCNs on different nodes is still underexplored. To address the limitations, we study GCNs with respect to the node degree distribution. We show that GCNs have a higher accuracy on nodes with larger degrees even if they are underrepresented in most graphs, with both empirical observation and theoretical proof. We then propose Self-Supervised-Learning Degree-Specific GCN (SL-DSGCN) which handles the degree-related biases of GCNs from model and data aspects. Firstly, we design a degree-specific GCN layer that models both discrepancies and similarities of nodes with different degrees, and reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we develop a self-supervised-learning algorithm that assigns pseudo labels with uncertainty scores on unlabeled nodes using a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. We further exploit uncertainty scores as dynamic weights on pseudo labels in the stochastic gradient descent for SL-DSGCN. We validate \ours on three benchmark datasets, and confirm SL-DSGCN not only outperforms state-of-the-art self-training/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.

研究动机与目标

• 解决尽管在现实世界图中低阶节点占比较低，但GCN在这些节点上表现不佳的问题。
• 探究为何GCN在高阶节点上表现出更高的准确率，并将性能差异与节点度数分布联系起来。
• 通过度数特定的网络架构和自监督学习，减少GCN中的数据级和模型级偏差。
• 利用不确定性感知的伪标签和动态加权策略，提升不同度数节点上GCN的泛化能力和公平性。

提出的方法

• 设计一种度数特定的GCN层，为不同度数的节点学习独立的消息传递机制，降低参数共享带来的偏差。
• 引入贝叶斯神经网络，为未标记节点生成带有不确定性评分的伪标签，增强其与已标记邻居的连接性。
• 将不确定性评分用作随机梯度下降中的动态权重，优先考虑可靠的伪标签进行训练。
• 应用自监督学习为未标记节点分配伪标签，增加其有效监督信号，减少数据级偏差。
• 结合度数特定的网络层与不确定性加权的伪标签生成方法，联合缓解GCN中的模型级与数据级偏差。
• 通过融合已标记数据和不确定性加权伪标签数据的损失函数，端到端训练SL-DSGCN。

实验结果

研究问题

• RQ1 为何尽管在大多数图中高阶节点出现频率较低，GCN在这些节点上仍表现出更高的准确率？
• RQ2 度数特定的消息传递机制在多大程度上能减少GCN中的模型级偏差？
• RQ3 不确定性感知的伪标签生成方法如何提升低阶节点的性能？
• RQ4 结合度数特定架构与带动态加权的自监督学习，是否能超越现有的自训练和自监督GCN方法？
• RQ5 SL-DSGCN如何提升不同度数节点之间的公平性与泛化能力？

主要发现

• 即使在图中低阶节点占比较低，GCN在这些节点上的准确率仍显著低于高阶节点。
• 所提出的SL-DSGCN模型在三个基准数据集上，平均将低阶节点的GCN准确率提升了高达15%。
• 度数特定的GCN层通过为每类度数分组独立参数化，降低了模型级偏差，从而提升了稀有但关键的低阶节点的性能。
• 不确定性感知的伪标签生成方法通过增强消息传递过程中低阶节点与已标记邻居的连接性，提升了其有效监督程度。
• 基于不确定性评分的动态加权策略带来了更稳定和准确的训练过程，尤其显著改善了低阶节点预测的收敛性。
• 在引文网络与社交网络基准数据集上，SL-DSGCN在整体准确率和低阶节点性能方面，均优于最先进的自训练与自监督GCN方法。