[论文解读] DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs

Mengjia Xu, Apoorva Singh+1 more|arXiv (Cornell University)|Sep 28, 2021

Human Mobility and Location-Based Analysis被引用 4

一句话总结

DynG2G 提出了一种随机图嵌入方法，将时间节点嵌入建模为时变的多变量正态分布，从而在动态图中实现不确定性量化。该方法在八个不同基准上的时间链接预测任务中达到最先进性能，同时揭示出一个普遍关系：最优嵌入维度 $L_o$ 等于内在不确定性维度 $D_u$，从而实现自适应维度选择。

ABSTRACT

Dynamic graph embedding has gained great attention recently due to its capability of learning low dimensional graph representations for complex temporal graphs with high accuracy. However, recent advances mostly focus on learning node embeddings as deterministic "vectors" for static graphs yet disregarding the key graph temporal dynamics and the evolving uncertainties associated with node embedding in the latent space. In this work, we propose an efficient stochastic dynamic graph embedding method (DynG2G) that applies an inductive feed-forward encoder trained with node triplet-based contrastive loss. Every node per timestamp is encoded as a time-dependent probabilistic multivariate Gaussian distribution in the latent space, hence we can quantify the node embedding uncertainty on-the-fly. We adopted eight different benchmarks that represent diversity in size (from 96 nodes to 87,626 and from 13,398 edges to 4,870,863) and diversity in dynamics. We demonstrate via extensive experiments on these eight dynamic graph benchmarks that DynG2G achieves new state-of-the-art performance in capturing the underlying temporal node embeddings. We also demonstrate that DynG2G can predict the evolving node embedding uncertainty, which plays a crucial role in quantifying the intrinsic dimensionality of the dynamical system over time. We obtain a universal relation of the optimal embedding dimension, $L_o$, versus the effective dimensionality of uncertainty, $D_u$, and we infer that $L_o=D_u$ for all cases. This implies that the uncertainty quantification approach we employ in the DynG2G correctly captures the intrinsic dimensionality of the dynamics of such evolving graphs despite the diverse nature and composition of the graphs at each timestamp. Moreover, this $L_0 - D_u$ correlation provides a clear path to select adaptively the optimum embedding size at each timestamp by setting $L \ge D_u$.

研究动机与目标

• 解决确定性节点嵌入在捕捉演化图中时间动态性和不确定性方面的局限性。
• 开发一种高效、可扩展且归纳性的动态图表示学习方法，支持不确定性量化。
• 通过将嵌入维度与动力系统的内在不确定性关联，实现自适应的维度选择。
• 提升在具有异构时间动态特性的图中的训练稳定性和计算效率。
• 在多样化的时间图基准上展示最先进的性能，并具备稳健的不确定性估计能力。

提出的方法

• DynG2G 使用一种随机的、归纳性的前馈编码器，独立处理每个图快照，将节点在潜在空间中编码为多变量正态分布。
• 从 k 跳邻域中采样节点三元组，并结合边权重以保留结构关系。
• 模型采用基于三元组能量的平方指数对比损失来优化节点表示，促使语义相似的节点在潜在空间中更接近。
• 使用前一时刻的超参数初始化每个新时间戳的编码器，以确保训练稳定性和归纳偏差。
• 通过根据有效不确定性维度 $D_u$ 动态调整潜在维度，支持自适应嵌入大小。
• 通过学习到的多变量正态分布的方差量化不确定性，实现实时不确定性监控和鲁棒性评估。

Figure 1: Schematic diagram of our proposed DynG2G framework for stochastic temporal graph embedding. The dynamic graph $G$ is composed of $T$ graph snapshots. To learn the latent graph representations for the $\mathcal{G}=\{G_{1},G_{2},...,G_{T}\}$ over $T$ timestamps (vertices and links in red are

实验结果

研究问题

• RQ1 随机图嵌入方法能否在捕捉演化图的时间动态性的同时，有效量化节点嵌入的不确定性？
• RQ2 在动态图中，最优嵌入维度 $L_o$ 与内在不确定性维度 $D_u$ 之间存在何种关系？
• RQ3 DynG2G 是否在多样化图基准上的时间链接预测任务中优于现有的确定性和随机方法？
• RQ4 DynG2G 中的不确定性量化能否指导每个时间戳的自适应嵌入维度选择？
• RQ5 DynG2G 如何在具有不同规模和动态特性的图中保持计算效率和训练稳定性？

主要发现

• DynG2G 在涵盖 96 至 87,626 个节点、13,398 至 4,870,863 条边的八个多样化基准上，实现了时间链接预测的最先进性能。
• 发现一个普遍关系：所有基准中均满足 $L_o = D_u$，表明不确定性量化正确捕捉了动力系统的内在维度。
• 模型在不同嵌入尺寸下保持一致的每轮训练计算成本，单张 GPU 上的平均训练时间从 Reality Mining 的 0.67 秒到 Facebook 的 60.5 秒不等。
• 不确定性估计在不同运行中平滑且与初始化无关，除 UCI 基准外，后者表现出不规则动态。
• 通过设定 $L \geq D_u$，该方法实现了自适应维度选择，为每个时间戳提供了最优嵌入尺寸的合理路径。
• DynG2G 展现出鲁棒性和可扩展性，大型图如 Digg 的 GPU 显存使用峰值为 5GB，而小型图如 Reality Mining 则保持在 2GB 以下。

Figure 2: Visualization of the evolving dynamics of two temporal graphs for two of our eight benchmarks: synthetic SBM dataset (left) and Digg (right). Top row: snapshots of the two datasets at different timestamps (indicated by the number in each image). Bottom row: Growth rate of edges as a functi