[Paper Review] Alleviating neighbor bias: augmenting graph self-supervise learning with structural equivalent positive samples

Jiawei Zhu, Hong Mei+2 more|arXiv (Cornell University)|Dec 8, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes a topological signal-driven self-supervised learning method to alleviate neighbor bias in graph neural networks by incorporating structurally equivalent non-neighboring nodes as positive samples. Using persistent homology to extract multiscale topological features, the method computes structural equivalence between node pairs and applies a topological loss to pull high-equivalence non-neighbors closer in representation space, significantly improving node classification accuracy across seven benchmark datasets.

ABSTRACT

In recent years, using a self-supervised learning framework to learn the general characteristics of graphs has been considered a promising paradigm for graph representation learning. The core of self-supervised learning strategies for graph neural networks lies in constructing suitable positive sample selection strategies. However, existing GNNs typically aggregate information from neighboring nodes to update node representations, leading to an over-reliance on neighboring positive samples, i.e., homophilous samples; while ignoring long-range positive samples, i.e., positive samples that are far apart on the graph but structurally equivalent samples, a problem we call "neighbor bias." This neighbor bias can reduce the generalization performance of GNNs. In this paper, we argue that the generalization properties of GNNs should be determined by combining homogeneous samples and structurally equivalent samples, which we call the "GC combination hypothesis." Therefore, we propose a topological signal-driven self-supervised method. It uses a topological information-guided structural equivalence sampling strategy. First, we extract multiscale topological features using persistent homology. Then we compute the structural equivalence of node pairs based on their topological features. In particular, we design a topological loss function to pull in non-neighboring node pairs with high structural equivalence in the representation space to alleviate neighbor bias. Finally, we use the joint training mechanism to adjust the effect of structural equivalence on the model to fit datasets with different characteristics. We conducted experiments on the node classification task across seven graph datasets. The results show that the model performance can be effectively improved using a strategy of topological signal enhancement.

Motivation & Objective

• To address neighbor bias in graph self-supervised learning, where models over-rely on neighboring (homophilous) positive samples and underutilize structurally equivalent distant nodes.
• To propose a method that identifies and leverages structurally equivalent node pairs—those with similar topological roles but not direct neighbors—as positive samples.
• To improve generalization in graph representation learning by combining homophilous and structurally equivalent positive samples, as formalized in the GC combination hypothesis.
• To develop a joint training mechanism that adapts the influence of structural equivalence based on dataset characteristics.

Proposed method

• Extract multiscale topological features using persistent homology, which captures the topological structure of nodes at multiple scales.
• Compute structural equivalence between node pairs based on their persistent homology-derived topological features.
• Design a topological loss function that encourages the representation space to pull together non-neighboring node pairs with high structural equivalence.
• Integrate the topological loss with standard contrastive learning via a joint training mechanism, controlled by a hyperparameter λ to balance homophily and structural equivalence.
• Use node degree and Ricci curvature as alternative assignment functions for persistent homology, with results showing robustness to the choice of function.
• Apply different resolution settings in persistent image generation to characterize topology at varying granularities, with performance evaluated across datasets of varying size and density.

Experimental results

Research questions

• RQ1 Can structurally equivalent non-neighboring nodes serve as effective positive samples in graph self-supervised learning, beyond homophilous neighbors?
• RQ2 How can topological features extracted via persistent homology be used to identify and measure structural equivalence between nodes?
• RQ3 Does incorporating structural equivalence improve generalization performance in graph neural networks compared to relying solely on neighborhood-based positive samples?
• RQ4 How does the choice of topological description (e.g., degree vs. Ricci curvature) affect model performance?
• RQ5 What is the optimal granularity for representing topological features in persistent homology for different graph characteristics?

Key findings

• The proposed method improved node classification accuracy on all seven benchmark datasets, with gains ranging from 1.6% to 2.6% over baseline models.
• On Cora, the model achieved 84.3% accuracy with λ=0.1, outperforming the baseline of 82.7%.
• On Amazon Photo, the method reached 93.7% accuracy with λ=0.3, exceeding the baseline of 92.2%.
• Using finer resolution in persistent image generation improved performance on small and sparse graphs like Cora and Citeseer, while coarser resolution worked better on large, dense graphs like Coauthor Physics.
• The method showed robustness to the choice of topological assignment function, with node degree and Ricci curvature yielding similar performance gains.
• Joint training with λ allowed the model to adaptively balance homophily and structural equivalence, leading to consistent improvements across diverse datasets.