[论文解读] Effective Stabilized Self-Training on Few-Labeled Graph Data

Ziang Zhou, Shi, Jieming+3 more|arXiv (Cornell University)|Oct 7, 2019

Advanced Graph Neural Networks参考文献 62被引用 6

一句话总结

该论文提出Stabilized Self-Training（SST），一种在极端标签稀缺条件下通过迭代添加高置信度伪标签并过滤低置信度标签来增强图神经网络（GNNs）的框架。SST提升了训练稳定性和准确性，在仅每类1个标注节点的Cora数据集上，测试准确率分别达到62.5%和66.4%，较GCN和DAGNN分别高出17.9%和6.6%。

ABSTRACT

Graph neural networks (GNNs) are designed for semi-supervised node classification on graphs where only a subset of nodes have class labels. However, under extreme cases when very few labels are available (e.g., 1 labeled node per class), GNNs suffer from severe performance degradation. Specifically, we observe that existing GNNs suffer from unstable training process on few-labeled graphs, resulting to inferior performance on node classification. Therefore, we propose an effective framework, Stabilized Self-Training (SST), which is applicable to existing GNNs to handle the scarcity of labeled data, and consequently, boost classification accuracy. We conduct thorough empirical and theoretical analysis to support our findings and motivate the algorithmic designs in SST. We apply SST to two popular GNN models GCN and DAGNN, to get SSTGCN and SSTDA methods respectively, and evaluate the two methods against 10 competitors over 5 benchmarking datasets. Extensive experiments show that the proposed SST framework is highly effective, especially when few labeled data are available. Our methods achieve superior performance under almost all settings over all datasets. For instance, on a Cora dataset with only 1 labeled node per class, the accuracy of SSTGCN is 62.5%, 17.9% higher than GCN, and the accuracy of SSTDA is 66.4%, which outperforms DAGNN by 6.6%.

研究动机与目标

• 解决当每类仅提供1个标注节点时GNNs出现的严重性能下降问题。
• 识别现有GNNs（如GCN、DAGNN）在极端标签稀缺条件下的不稳定训练动态。
• 开发一种稳定且可泛化的自训练框架，可应用于现有GNNs，以在少样本设置下提升节点分类性能。
• 通过过滤低置信度预测，缓解标准自训练中常见的伪标签不平衡问题。

提出的方法

• 提出一种稳定自训练框架（SST），选择性地将高置信度预测标签作为伪标签添加到训练集中。
• 定义置信度阈值η，用于过滤低置信度预测，确保仅使用可靠的伪标签。
• 理论分析表明，高置信度伪标签可通过平衡标签分布来降低分类误差。
• 将SST应用于GCN和DAGNN，得到SSTGCN和SSTDA，显著提升了训练稳定性与泛化能力。
• 对未标注节点采用均匀采样，推导标签预测置信度的概率边界。
• 利用理论边界（引理1–3）确保所选伪标签在统计上可靠，从而减少误差传播。

实验结果

研究问题

• RQ1 为何现有GNNs（如GCN和DAGNN）在极端标签稀缺条件下（如每类仅1个标注节点）会失效？
• RQ2 当仅有少量标注节点时，GNNs中的不稳定训练表现为何种形式？
• RQ3 是否可以通过稳定自训练在不改变网络架构的前提下提升低标签场景下的性能？
• RQ4 何种标准可确保伪标签不会因不平衡或低置信度而降低性能？
• RQ5 所提出的SST框架在准确率与稳定性方面相较于现有自训练和GNN方法表现如何？

主要发现

• 在每类仅1个标注节点的Cora数据集中，SSTGCN达到62.5%的测试准确率，较GCN提升17.9%。
• SSTDA在相同设置下准确率达到66.4%，较DAGNN高出6.6%。
• 实验结果表明，SST显著降低了不同运行之间预测标签分布的方差，表明训练稳定性得到提升。
• 理论分析证实，高置信度伪标签通过确保正确预测概率的下限，有效降低分类误差。
• SST在5个基准数据集和10种对比方法中均保持优越性能，尤其在低标签场景下表现突出。
• 过滤低置信度伪标签可有效防止误差传播，并缓解标准自训练中常见的标签不平衡问题。