[Paper Review] Self-Supervised Dynamic Graph Representation Learning via Temporal Subgraph Contrast

Linpu Jiang, Kejia Chen+1 more|arXiv (Cornell University)|Dec 16, 2021

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes DySubC, a self-supervised dynamic graph representation learning framework that enhances node embeddings by contrasting temporal subgraphs—subgraphs sampled using both structural and timestamped edge information. By jointly optimizing structural and temporal contrastive losses, DySubC achieves state-of-the-art performance in link prediction across five real-world datasets, demonstrating the critical role of temporal dynamics in improving representation quality.

ABSTRACT

Self-supervised learning on graphs has recently drawn a lot of attention due to its independence from labels and its robustness in representation. Current studies on this topic mainly use static information such as graph structures but cannot well capture dynamic information such as timestamps of edges. Realistic graphs are often dynamic, which means the interaction between nodes occurs at a specific time. This paper proposes a self-supervised dynamic graph representation learning framework (DySubC), which defines a temporal subgraph contrastive learning task to simultaneously learn the structural and evolutional features of a dynamic graph. Specifically, a novel temporal subgraph sampling strategy is firstly proposed, which takes each node of the dynamic graph as the central node and uses both neighborhood structures and edge timestamps to sample the corresponding temporal subgraph. The subgraph representation function is then designed according to the influence of neighborhood nodes on the central node after encoding the nodes in each subgraph. Finally, the structural and temporal contrastive loss are defined to maximize the mutual information between node representation and temporal subgraph representation. Experiments on five real-world datasets demonstrate that (1) DySubC performs better than the related baselines including two graph contrastive learning models and four dynamic graph representation learning models in the downstream link prediction task, and (2) the use of temporal information can not only sample more effective subgraphs, but also learn better representation by temporal contrastive loss.

Motivation & Objective

• To address the limitation of static graph representation learning in capturing temporal evolution of dynamic graphs.
• To develop a self-supervised framework that jointly models structural and temporal dynamics in dynamic graphs.
• To improve node representation quality by incorporating edge timestamps into subgraph sampling and contrastive learning.
• To validate that temporal information enhances subgraph sampling effectiveness and downstream representation performance.

Proposed method

• A novel temporal subgraph sampling strategy is proposed, using each node as a center and sampling neighbors based on both proximity and edge timestamps.
• A time-aware GNN encoder processes each sampled temporal subgraph to generate node and subgraph representations.
• A time-weighted readout function computes a summary of the temporal subgraph, emphasizing recent interactions.
• Structural and temporal contrastive losses are defined to maximize mutual information between node representations and their corresponding temporal subgraph representations.
• The model is trained using positive pairs (central node and its time-weighted subgraph), structural negative pairs (other subgraphs), and temporal negative pairs (unweighted subgraph).
• Hyperparameters α and β balance structural and temporal information in sampling and representation, respectively.

Experimental results

Research questions

• RQ1 Can incorporating edge timestamps into subgraph sampling improve the quality of dynamic graph representations?
• RQ2 Does temporal contrastive learning lead to better node representation learning compared to static or purely structural contrastive methods?
• RQ3 How do subgraph size and hyperparameters α and β affect the performance of the proposed framework?
• RQ4 To what extent does temporal information enhance link prediction performance in dynamic graphs?

Key findings

• DySubC outperforms four dynamic graph learning models and two graph contrastive learning models in link prediction across five real-world datasets.
• On the fb-forum dataset, DySubC achieves a 0.05 AUC score improvement over the base model when the time-enhanced module is enabled.
• Subgraph size analysis shows optimal performance at size 20 across all datasets, with larger sizes (e.g., 100) degrading performance on smaller graphs like fb-forum.
• Sensitivity analysis reveals that α = 10 and β = 1.6 yield the best performance, indicating that both structural and temporal information are crucial in subgraph sampling and representation.
• t-SNE visualization confirms that DySubC better clusters nodes based on recent interactions, while Sub-Con fails to capture such temporal proximity.
• The model demonstrates robustness to hyperparameter perturbations around median values, suggesting stable learning dynamics when temporal and structural signals are balanced.