Skip to main content
QUICK REVIEW

[论文解读] Do We Really Need Complicated Model Architectures For Temporal Networks?

Weilin Cong, Si Zhang|arXiv (Cornell University)|Feb 22, 2023
Advanced Graph Neural NetworksComputer Science被引用 19
一句话总结

GraphMixer 是一个简单的基于 MLP 的架构,具有固定时间编码和1跳邻居信息,在没有 RNNs 或自注意力的情况下实现了最先进的时序链接预测,并且收敛更快、泛化能力更好。

ABSTRACT

Recurrent neural network (RNN) and self-attention mechanism (SAM) are the de facto methods to extract spatial-temporal information for temporal graph learning. Interestingly, we found that although both RNN and SAM could lead to a good performance, in practice neither of them is always necessary. In this paper, we propose GraphMixer, a conceptually and technically simple architecture that consists of three components: (1) a link-encoder that is only based on multi-layer perceptrons (MLP) to summarize the information from temporal links, (2) a node-encoder that is only based on neighbor mean-pooling to summarize node information, and (3) an MLP-based link classifier that performs link prediction based on the outputs of the encoders. Despite its simplicity, GraphMixer attains an outstanding performance on temporal link prediction benchmarks with faster convergence and better generalization performance. These results motivate us to rethink the importance of simpler model architecture.

研究动机与目标

  • 评估 RNNs 和自注意力在时序图学习中是否不可或缺。
  • 提出一个简单的、无 RNN/SAM 的架构(GraphMixer),并在时序链接预测任务上评估其性能。
  • 识别促成 GraphMixer 成功的因素,包括输入简洁性、固定时间编码以及基于 MLP 的链接编码器。

提出的方法

  • 引入 GraphMixer,包含三个组件:链接编码器(基于 MLP)、节点编码器(邻居均值汇聚)以及基于 MLP 的链接分类器。
  • 实现一个固定时间编码函数 z(t) = cos(t * omega) 来区分时间戳,其中 omega 是固定的且不可学习。
  • 在时间编码之后使用一层 MLP-Mixer 来总结时序链接信息,并对其进行零填充以达到固定长度 K。
  • 通过节点特征和 1 跳邻居均值汇聚来表示节点信息;将其与链接编码器的输出结合用于链接预测。
  • 在多个真实时序图数据集上进行训练和评估,与基于 RNN 和 SAM 的基线以及消融版本 GraphMixer-L(仅链接编码器)和 GraphMixer-N(仅节点编码器)进行对比。
Figure 1: (Left) Temporal graph with nodes $v_{1},\ldots,v_{5}$ , per-link timestamps $t_{1},\ldots,t_{6}$ indicate when two nodes interact. For example, $v_{1},v_{2}$ interact at $t_{1},t_{5}$ . (Right) Each node has its node features (e.g., $\mathbf{x}_{1}^{\text{node}}$ for $v_{1}$ ) and each tem
Figure 1: (Left) Temporal graph with nodes $v_{1},\ldots,v_{5}$ , per-link timestamps $t_{1},\ldots,t_{6}$ indicate when two nodes interact. For example, $v_{1},v_{2}$ interact at $t_{1},t_{5}$ . (Right) Each node has its node features (e.g., $\mathbf{x}_{1}^{\text{node}}$ for $v_{1}$ ) and each tem

实验结果

研究问题

  • RQ1RNNs 和 SAMs 在时序图学习中是否始终必需?
  • RQ2一个不含 RNNs/SAMs 的简单基于 MLP 的架构是否也能在时序链接预测上达到具有竞争力甚至更优的性能?
  • RQ3哪些因素促成 GraphMixer 的性能提升(输入简洁性、固定时间编码、编码器设计)?

主要发现

  • GraphMixer 在所有数据集上都在时序链接预测的平均精度方面优于所有基线。
  • 固定时间编码函数相比基线使用的可训练时间编码,带来更平滑的训练和更好的泛化。
  • 基于 MLP 的链接编码器能够有效捕捉时序序列,并从 1 跳邻居信息中获益;单独的节点编码器则不足。
  • 在某些数据集上,one-hot 节点身份信息可以优于节点特征,凸显了节点身份信息的重要性。
  • 在默认设置下,具有大量超参数的更复杂模型(如 CAWs、TGSRec、DDGCL)不一定表现更好。
Figure 2: (a) Time-encoding function that pre-process timestamp $t$ into a vector $\cos(t\bm{\omega})$ . The x-axis is the vector dimension and the y-axis is the cosine value. (b) link-encoder takes the temporal link information of node $v_{2}$ as inputs and outputs a vector $\mathbf{t}_{2}(t_{0})$
Figure 2: (a) Time-encoding function that pre-process timestamp $t$ into a vector $\cos(t\bm{\omega})$ . The x-axis is the vector dimension and the y-axis is the cosine value. (b) link-encoder takes the temporal link information of node $v_{2}$ as inputs and outputs a vector $\mathbf{t}_{2}(t_{0})$

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。