[Paper Review] DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs

Mengjia Xu, Apoorva Singh+1 more|arXiv (Cornell University)|Sep 28, 2021

Human Mobility and Location-Based Analysis4 citations

TL;DR

DynG2G proposes a stochastic graph embedding method that models temporal node embeddings as time-dependent multivariate Gaussian distributions, enabling uncertainty quantification in dynamic graphs. It achieves state-of-the-art performance in temporal link prediction across eight diverse benchmarks while revealing a universal relationship where optimal embedding dimension $L_o$ equals intrinsic uncertainty dimension $D_u$, enabling adaptive dimension selection.

ABSTRACT

Dynamic graph embedding has gained great attention recently due to its capability of learning low dimensional graph representations for complex temporal graphs with high accuracy. However, recent advances mostly focus on learning node embeddings as deterministic "vectors" for static graphs yet disregarding the key graph temporal dynamics and the evolving uncertainties associated with node embedding in the latent space. In this work, we propose an efficient stochastic dynamic graph embedding method (DynG2G) that applies an inductive feed-forward encoder trained with node triplet-based contrastive loss. Every node per timestamp is encoded as a time-dependent probabilistic multivariate Gaussian distribution in the latent space, hence we can quantify the node embedding uncertainty on-the-fly. We adopted eight different benchmarks that represent diversity in size (from 96 nodes to 87,626 and from 13,398 edges to 4,870,863) and diversity in dynamics. We demonstrate via extensive experiments on these eight dynamic graph benchmarks that DynG2G achieves new state-of-the-art performance in capturing the underlying temporal node embeddings. We also demonstrate that DynG2G can predict the evolving node embedding uncertainty, which plays a crucial role in quantifying the intrinsic dimensionality of the dynamical system over time. We obtain a universal relation of the optimal embedding dimension, $L_o$, versus the effective dimensionality of uncertainty, $D_u$, and we infer that $L_o=D_u$ for all cases. This implies that the uncertainty quantification approach we employ in the DynG2G correctly captures the intrinsic dimensionality of the dynamics of such evolving graphs despite the diverse nature and composition of the graphs at each timestamp. Moreover, this $L_0 - D_u$ correlation provides a clear path to select adaptively the optimum embedding size at each timestamp by setting $L \ge D_u$.

Motivation & Objective

• To address the limitations of deterministic node embeddings in capturing temporal dynamics and uncertainty in evolving graphs.
• To develop an efficient, scalable, and inductive method for dynamic graph representation learning that supports uncertainty quantification.
• To enable adaptive selection of embedding dimension by linking it to the intrinsic uncertainty of the dynamical system.
• To improve training stability and computational efficiency in graphs with heterogeneous temporal dynamics.
• To demonstrate state-of-the-art performance on diverse temporal graph benchmarks with robust uncertainty estimation.

Proposed method

• DynG2G uses a stochastic, inductive feed-forward encoder that processes each graph snapshot independently, encoding nodes as multivariate Gaussian distributions in the latent space.
• Node triplets are sampled from k-hop neighborhoods with edge-weighted proximity to preserve structural relationships.
• The model employs a triplet energy-based square-exponential contrastive loss to optimize node representations, promoting semantically similar nodes to be closer in latent space.
• Hyperparameters from previous timestamps are used to initialize the encoder at each new timestamp, ensuring training stability and inductive bias.
• The method supports adaptive embedding size by dynamically adjusting the latent dimension based on the effective uncertainty dimension $D_u$.
• Uncertainty is quantified via the variance of the learned multivariate Gaussian, enabling real-time uncertainty monitoring and robustness assessment.

Figure 1: Schematic diagram of our proposed DynG2G framework for stochastic temporal graph embedding. The dynamic graph $G$ is composed of $T$ graph snapshots. To learn the latent graph representations for the $\mathcal{G}=\{G_{1},G_{2},...,G_{T}\}$ over $T$ timestamps (vertices and links in red are

Experimental results

Research questions

• RQ1 Can a stochastic graph embedding method effectively capture temporal dynamics while quantifying node embedding uncertainty in evolving graphs?
• RQ2 What is the relationship between the optimal embedding dimension $L_o$ and the intrinsic uncertainty dimension $D_u$ in dynamic graphs?
• RQ3 Does DynG2G outperform existing deterministic and stochastic methods in temporal link prediction across diverse graph benchmarks?
• RQ4 Can uncertainty quantification in DynG2G guide adaptive selection of embedding dimension at each timestamp?
• RQ5 How does DynG2G maintain computational efficiency and training stability across graphs with varying sizes and dynamics?

Key findings

• DynG2G achieves state-of-the-art performance in temporal link prediction across eight diverse benchmarks, ranging from 96 to 87,626 nodes and 13,398 to 4,870,863 edges.
• A universal relationship is discovered: $L_o = D_u$ for all benchmarks, indicating that uncertainty quantification correctly captures the intrinsic dimensionality of the dynamical system.
• The model maintains consistent computational cost per epoch across different embedding sizes, with average training times ranging from 0.67s (Reality Mining) to 60.5s (Facebook) on a single GPU.
• Uncertainty estimates are smooth and initialization-independent across runs, except for the UCI benchmark, which exhibits irregular dynamics.
• The method enables adaptive dimension selection by setting $L \geq D_u$, providing a principled path to optimal embedding size at each timestamp.
• DynG2G demonstrates robustness and scalability, with GPU memory usage peaking at 5GB for large graphs like Digg, while remaining under 2GB for smaller ones like Reality Mining.

Figure 2: Visualization of the evolving dynamics of two temporal graphs for two of our eight benchmarks: synthetic SBM dataset (left) and Digg (right). Top row: snapshots of the two datasets at different timestamps (indicated by the number in each image). Bottom row: Growth rate of edges as a functi