[Paper Review] GraphSHA: Synthesizing Harder Samples for Class-Imbalanced Node Classification

Wen-Zhi Li, Chang‐Dong Wang+2 more|arXiv (Cornell University)|Jun 16, 2023

Advanced Graph Neural Networks4 citations

TL;DR

GraphSHA proposes a novel data augmentation framework that synthesizes harder minority-class samples in graph-structured data to alleviate the 'squeezed minority' problem in class-imbalanced node classification. By leveraging a semi-supervised mixup strategy (SemiMixup) to generate hard samples from anchor nodes and neighbor-class auxiliary nodes, GraphSHA enlarges minority decision boundaries without degrading neighbor classes, achieving state-of-the-art performance across seven benchmark datasets with multiple GNN backbones.

ABSTRACT

Class imbalance is the phenomenon that some classes have much fewer instances than others, which is ubiquitous in real-world graph-structured scenarios. Recent studies find that off-the-shelf Graph Neural Networks (GNNs) would under-represent minor class samples. We investigate this phenomenon and discover that the subspaces of minor classes being squeezed by those of the major ones in the latent space is the main cause of this failure. We are naturally inspired to enlarge the decision boundaries of minor classes and propose a general framework GraphSHA by Synthesizing HArder minor samples. Furthermore, to avoid the enlarged minor boundary violating the subspaces of neighbor classes, we also propose a module called SemiMixup to transmit enlarged boundary information to the interior of the minor classes while blocking information propagation from minor classes to neighbor classes. Empirically, GraphSHA shows its effectiveness in enlarging the decision boundaries of minor classes, as it outperforms various baseline methods in class-imbalanced node classification with different GNN backbone encoders over seven public benchmark datasets. Code is avilable at https://github.com/wenzhilics/GraphSHA.

Motivation & Objective

• To address the 'squeezed minority' problem in class-imbalanced node classification, where minor class subspaces are compressed in the latent space.
• To improve GNN performance on minority classes by synthesizing harder training samples that expand decision boundaries.
• To prevent interference from neighbor classes during synthesis by blocking information flow from minority to neighbor subspaces.
• To develop a generalizable, GNN-agnostic framework that enhances minority class representation without modifying loss functions.

Proposed method

• GraphSHA identifies hard minor samples near the decision boundary as anchor nodes for synthesis.
• It introduces a SemiMixup module that generates synthetic samples via convex combination: $\mathbf{X}_{\text{syn}} = \delta \mathbf{X}_{\text{anc}} + (1 - \delta) \mathbf{X}_{\text{aux}}$, where $\mathbf{X}_{\text{anc}}$ is a hard minor sample and $\mathbf{X}_{\text{aux}}$ is a neighbor-class node.
• The mixing coefficient $\delta$ is sampled from a distribution biased toward smaller values to favor harder samples.
• The method ensures that synthesized features remain within the minor class subspace while preventing over-invasion into neighbor class regions.
• It uses a topological-aware sampling strategy to select auxiliary nodes from the neighbor class of the anchor node.
• The framework is compatible with any GNN backbone and operates in the latent space without altering the original graph structure.

Experimental results

Research questions

• RQ1 Can synthesizing harder minority samples effectively enlarge the decision boundary of minor classes in the latent space of GNNs?
• RQ2 How can we prevent the synthesis of harder samples from invading or degrading the subspaces of neighboring classes?
• RQ3 What is the optimal distribution for the mixing coefficient $\delta$ in the SemiMixup module to balance hardness and generalization?
• RQ4 Does GraphSHA outperform existing data augmentation and loss-modification baselines on both manually and naturally imbalanced graph benchmarks?
• RQ5 To what extent does GraphSHA mitigate the 'squeezed minority' problem as measured by misclassification distribution?

Key findings

• GraphSHA significantly reduces the misclassification rate of minority classes, with the probability of misclassified samples being minor classes approaching 0.5, indicating effective boundary enlargement.
• On Cora-LT and CiteSeer-LT with GCN, GraphSHA achieves the highest F1-score among all baselines, outperforming even advanced methods like TAM and ReNode.
• The SemiMixup module prevents neighbor class degradation, as evidenced by stable performance on major classes despite aggressive boundary expansion.
• Hyperparameter analysis shows that smaller $\mathbb{E}(\delta)$ leads to better performance, confirming that harder samples are more effective for boundary expansion.
• Visualization via t-SNE confirms that GraphSHA successfully enlarges minor class subspaces and includes more test-set minority nodes within their latent regions.
• The method generalizes across three GNN backbones (GCN, GAT, GraphSAGE) and seven public datasets, demonstrating robustness and scalability.