[Paper Review] Distance-wise Graph Contrastive Learning.

Deli Chen, Yankai Lin+5 more|arXiv (Cornell University)|Dec 14, 2020

Advanced Graph Neural Networks20 references4 citations

TL;DR

This paper proposes Distance-wise Graph Contrastive Learning (DwGCL), a method that adaptively applies contrastive learning to graph neural networks by considering each node’s topological distance from labeled nodes. By leveraging Group PageRank to identify information-poor, distant nodes and adjusting sampling strategies accordingly, DwGCL improves performance across five benchmark datasets and multiple GNN architectures under various annotation settings.

ABSTRACT

Contrastive learning (CL) has proven highly effective in graph-based semi-supervised learning (SSL), since it can efficiently supplement the limited task information from the annotated nodes in graph. However, existing graph CL (GCL) studies ignore the uneven distribution of task information across graph caused by the graph topology and the selection of annotated nodes. They apply CL to the whole graph evenly, which results in an incongruous combination of CL and graph learning. To address this issue, we propose to apply CL in the graph learning adaptively by taking the received task information of each node into consideration. Firstly, we introduce Group PageRank to measure the node information gain from graph and find that CL mainly works for nodes that are topologically far away from the labeled nodes. We then propose our Distance-wise Graph Contrastive Learning (DwGCL) method from two views:(1) From the global view of the task information distribution across the graph, we enhance the CL effect on nodes that are topologically far away from labeled nodes; (2) From the personal view of each node's received information, we measure the relative distance between nodes and then we adapt the sampling strategy of GCL accordingly. Extensive experiments on five benchmark graph datasets show that DwGCL can bring a clear improvement over previous GCL methods. Our analysis on eight graph neural network with various types of architecture and three different annotation settings further demonstrates the generalizability of DwGCL.

Motivation & Objective

• Address the uneven distribution of task information in graph-based semi-supervised learning due to graph topology and labeled node selection.
• Overcome the limitation of existing graph contrastive learning (GCL) methods that apply contrastive learning uniformly across the entire graph.
• Improve GCL effectiveness by focusing on nodes that are topologically distant from labeled nodes, where information gain is most needed.
• Develop an adaptive sampling strategy that considers each node’s relative distance to labeled nodes to enhance contrastive learning efficiency.
• Demonstrate the generalizability of the proposed method across diverse GNN architectures and annotation settings.

Proposed method

• Introduce Group PageRank to measure the information gain of each node from the graph structure, identifying nodes that are topologically distant from labeled nodes.
• Use Group PageRank scores to guide the selection of negative samples in contrastive learning, prioritizing nodes with low information gain.
• Adapt the sampling strategy of GCL based on the relative distance between nodes and labeled nodes, favoring distant nodes for contrastive augmentation.
• Apply contrastive learning in a distance-aware manner: enhance the contrastive signal for nodes far from labeled nodes, where supervision is sparse.
• Integrate the distance-aware sampling into the standard GCL framework without modifying the underlying GNN architecture.
• Leverage the global distribution of task information to reweight or reconfigure the contrastive learning objective based on topological distance.

Experimental results

Research questions

• RQ1 How does the topological distance between a node and labeled nodes affect its information gain in graph-based semi-supervised learning?
• RQ2 Can adaptive contrastive learning that prioritizes distant nodes improve performance compared to uniform GCL application?
• RQ3 How does the proposed distance-aware sampling strategy affect the generalization of GNNs across different architectures?
• RQ4 To what extent does DwGCL improve performance under varying levels of label scarcity and different label placement strategies?
• RQ5 Does the use of Group PageRank as a proxy for information gain lead to more effective contrastive learning in graphs?

Key findings

• DwGCL achieves consistent performance improvements over previous GCL methods on five benchmark graph datasets.
• The method is effective across multiple GNN architectures, demonstrating strong generalizability.
• The improvement is particularly significant for nodes that are topologically distant from labeled nodes, confirming the method’s focus on information-poor regions.
• Experiments under three different annotation settings show that DwGCL maintains superior performance regardless of label distribution.
• The use of Group PageRank successfully identifies nodes that benefit most from contrastive learning, validating its role in guiding adaptive sampling.
• The adaptive sampling strategy based on relative distance leads to more effective contrastive signal propagation in graphs with uneven label distribution.