[Paper Review] Self-Supervised Graph Representation Learning via Global Context Prediction

Zhen Peng, Yixiang Dong+3 more|arXiv (Cornell University)|Mar 3, 2020

Advanced Graph Neural NetworksComputer Science33 references39 citations

TL;DR

The paper proposes S2GRL, a self-supervised framework that learns global-context–aware node embeddings by predicting hop-based contextual positions between node pairs, outperforming many unsupervised methods and rivaling some supervised models.

ABSTRACT

To take full advantage of fast-growing unlabeled networked data, this paper introduces a novel self-supervised strategy for graph representation learning by exploiting natural supervision provided by the data itself. Inspired by human social behavior, we assume that the global context of each node is composed of all nodes in the graph since two arbitrary entities in a connected network could interact with each other via paths of varying length. Based on this, we investigate whether the global context can be a source of free and effective supervisory signals for learning useful node representations. Specifically, we randomly select pairs of nodes in a graph and train a well-designed neural net to predict the contextual position of one node relative to the other. Our underlying hypothesis is that the representations learned from such within-graph context would capture the global topology of the graph and finely characterize the similarity and differentiation between nodes, which is conducive to various downstream learning tasks. Extensive benchmark experiments including node classification, clustering, and link prediction demonstrate that our approach outperforms many state-of-the-art unsupervised methods and sometimes even exceeds the performance of supervised counterparts.

Motivation & Objective

• Motivate learning node representations from unlabeled graph data using natural supervision inherent in graph structure.
• Propose a self-supervised framework that predicts relative contextual positions (hop counts) between node pairs to encode global topology.
• Demonstrate that hop-based supervision yields representations competitive with state-of-the-art unsupervised methods and some supervised baselines.

Proposed method

• Define hop-based global context for each node as nodes reachable within k hops, partitioned into major categories (e.g., 1-hop, 2-hop, etc.).
• Train an encoder f_ω to produce node embeddings such that a classifier h_θ predicts the hop-based context between node pairs from their embeddings.
• Use a symmetric interaction proxy (absolute difference) ⟨z_i, z_j⟩ = |z_i − z_j| to ensure permutation symmetry in the context predictions.
• Optimize a cross-class objective over major context categories to learn global-context-aware representations.
• Employ batch sampling to address computational and class-imbalance challenges in large graphs.
• Explore hyperparameters for major-class construction to balance discriminability and generalization.

Experimental results

Research questions

• RQ1 Can global graph topology provide a free supervisory signal for self-supervised graph representation learning?
• RQ2 Does predicting hop-based contextual positions between node pairs yield embeddings that capture global structure and improve downstream tasks?
• RQ3 How does the construction of major context categories affect embedding quality?
• RQ4 How does S2GRL compare to existing unsupervised and supervised graph representation methods on standard benchmarks?

Key findings

• S2GRL achieves 83.7% (Cora), 72.1% (Citeseer), and 82.4% (Pubmed) in transductive node classification, outperforming many unsupervised baselines.
• In inductive classification, S2GRL reaches 66.0% (PPI) and 95.0% (Reddit), surpassing several baselines.
• For clustering (NMI), S2GRL attains 0.540 (Cora), 0.432 (Citeseer), and 0.332 (Pubmed), competitive with existing methods.
• In link prediction, S2GRL achieves 80.4–78.2% AUC on BlogCatalog and 91.4–89.8% AUC on Flickr across varying edge-removal rates, outperforming several baselines.
• Visual analyses (t-SNE) indicate learned embeddings reflect topological distances, supporting the global-context hypothesis.
• Using 1-hop, 2-hop, and 3-hop as distinct major categories yields better representations than overly fine-grained partitioning.