[论文解读] Inductive Representation Learning on Temporal Graphs

Da Xu, Chuanwei Ruan+3 more|arXiv (Cornell University)|Feb 19, 2020

Advanced Graph Neural Networks参考文献 32被引用 32

一句话总结

介绍 TGAT，一种基于 Bochner 定理的函数时间编码来生成对演化图中时变任务的时间感知节点嵌入的时序图注意网络。

ABSTRACT

Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal patterns. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.

研究动机与目标

• 解决对在演化图上进行归纳性、时间敏感嵌入的需求。
• 在节点表示中同时捕捉时间模式与拓扑上下文。
• 在未见节点和未见时间点上实现准确的节点分类和连边预测。
• 提供可扩展、可并行的架构，适用于工业规模的时序图。

提出的方法

• 提出 Temporal Graph Attention (TGAT) 层，通过自注意力聚合时序邻域特征。
• 用基于 Bochner 定理的函数时间编码替代传统的位置编码来建模时间。
• 将时间核 K(t1,t2) 定义为时间编码的内积，并用 cosine 和 sine 的蒙特卡洛采样来近似。
• 将 TGAT 扩展以在消息传递中加入边特征。
• 堆叠 TGAT 层以捕捉多跳的时序依赖性，并在稳定性和性能方面可选地使用多头注意力。

实验结果

研究问题

• RQ1 在归纳设置下，我们如何学习时变图的时间感知节点嵌入？
• RQ2 基于自注意力的架构结合函数时间编码，是否能在归纳和传导任务上超越现有的时序图嵌入？
• RQ3 时间特征交互如何影响在演化图上的节点分类和连边预测？

主要发现

• TGAT 在三个数据集（Reddit、Wikipedia、Industrial）上在传导和归纳任务中均取得强劲表现。
• 通过 Bochner 基于函数时间编码学习的时间核使注意力中的时间特征交互成为可能，从而提升嵌入质量。
• 在多头注意力和时间特征整合下，TGAT 在关键指标上超越包括 GAE、VGAE、CDTNE、GAT 和 GraphSAGE 变体等基线。
• 该方法通过一次前向传播即可对未见节点和新的时间点进行归纳推断，实现可扩展部署。