[Paper Review] Toward Robust Graph Semi-Supervised Learning against Extreme Data Scarcity

Kaize Ding, Elnaz Nouri+3 more|arXiv (Cornell University)|Aug 26, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes AGST, a novel graph self-training framework that enhances robustness in semi-supervised node classification under extreme data scarcity by introducing structural and semantic data augmentation modules. By decoupling message passing and using a topology augmentation function to reduce noise and improve long-range feature propagation, AGST achieves state-of-the-art performance, with gains of up to 3.2% accuracy over baselines on low-shot benchmarks.

ABSTRACT

The success of graph neural networks on graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only few labeled nodes are available, how to improve their robustness is a key to achieve replicable and sustainable graph semi-supervised learning. Though self-training has been shown to be powerful for semi-supervised learning, its application on graph-structured data may fail because (1) larger receptive fields are not leveraged to capture long-range node interactions, which exacerbates the difficulty of propagating feature-label patterns from labeled nodes to unlabeled nodes; and (2) limited labeled data makes it challenging to learn well-separated decision boundaries for different node classes without explicitly capturing the underlying semantic structure. To address the challenges of capturing informative structural and semantic knowledge, we propose a new graph data augmentation framework, AGST (Augmented Graph Self-Training), which is built with two new (i.e., structural and semantic) augmentation modules on top of a decoupled GST backbone. In this work, we investigate whether this novel framework can learn a robust graph predictive model under the low-data context. We conduct comprehensive evaluations on semi-supervised node classification under different scenarios of limited labeled-node data. The experimental results demonstrate the unique contributions of the novel data augmentation framework for node classification with few labeled data.

Motivation & Objective

• Address the challenge of poor generalization in graph semi-supervised learning when only a few labeled nodes are available.
• Overcome the structural bottleneck caused by over-squashing and noisy graph structures in long-range message passing.
• Mitigate the semantic bottleneck from unreliable pseudo-labels in low-data regimes by capturing well-separated decision boundaries.
• Design a robust framework that enhances both structural and semantic knowledge utilization during self-training.
• Improve model robustness and performance in real-world scenarios with limited human-annotated data.

Proposed method

• Proposes a decoupled GNN backbone where the teacher model performs high-order label propagation using Personalized PageRank to generate pseudo-labels.
• Introduces a structural data augmentation module that applies a deterministic topology refinement function in each GST iteration to reduce noise and improve long-range information flow.
• Employs a weakly-supervised contrastive loss to enhance semantic knowledge by encouraging feature clustering of nodes with the same pseudo-label.
• Uses a topology augmentation function that dynamically adjusts the graph structure by adding/removing edges based on learned model confidence, with hyper-parameters βₐ and βᵣ controlling edge addition and removal ratios.
• Combines cross-entropy loss on pseudo-labeled nodes and contrastive loss in a weighted objective function with hyper-parameters λ₁ and λ₂ to balance supervision and contrastive signals.
• Integrates both structural and semantic augmentation into a unified AGST framework to improve generalization under extreme data scarcity.

Figure 1. Overview of the proposed framework. In each training iteration, AGST first perform high-order label propagation to generate pseudo labels on unlabeled nodes, then conduct feature transformation with the augmented training label set. In the meantime, a weakly-supervised contrastive loss is

Experimental results

Research questions

• RQ1 Can structural data augmentation that refines graph topology improve long-range message passing and reduce over-squashing in low-data graph semi-supervised learning?
• RQ2 Can weakly-supervised contrastive learning enhance decision boundary separation when labeled data is extremely limited?
• RQ3 How do the proposed structural and semantic augmentation components contribute individually and jointly to model robustness under data scarcity?
• RQ4 What hyper-parameter settings (e.g., λ₁, λ₂, βₐ, βᵣ) yield optimal performance across different low-shot graph benchmarks?
• RQ5 Does AGST outperform existing GST baselines in terms of accuracy and stability under extreme data scarcity?

Key findings

• AGST achieves state-of-the-art performance on Cora and CiteSeer, with a 3.2% accuracy gain over the strongest baseline in the 5-shot setting.
• On Cora (5-shot), AGST achieves 82.57±0.22% accuracy, outperforming the next best method by 1.8%.
• On CiteSeer (5-shot), AGST reaches 71.52±0.11% accuracy, a 1.1% improvement over the baseline.
• The ablation study confirms that both the weakly-supervised contrastive loss and topology augmentation function are essential, with performance dropping significantly when either is removed.
• The optimal hyper-parameter values are λ₁=1 and λ₂=0.1, with βₐ≈0.4 and βᵣ≈0.2 yielding peak performance on Cora.
• Topology augmentation improves performance by up to 2.5% on noisy graphs like CiteSeer, demonstrating its effectiveness in mitigating structural noise.

(a) Cora