[论文解读] Continuous Graph Neural Networks

Louis-Pascal Xhonneux, Meng Qu+1 more|arXiv (Cornell University)|Dec 2, 2019

Advanced Graph Neural Networks参考文献 36被引用 41

一句话总结

CGNN 通过 ODEs 提供连续时间图神经网络来建模节点表示，解决过平滑问题并实现深层传播，在基准图上的节点分类结果强劲。

ABSTRACT

This paper builds on the connection between graph neural networks and traditional dynamical systems. We propose continuous graph neural networks (CGNN), which generalise existing graph neural networks with discrete dynamics in that they can be viewed as a specific discretisation scheme. The key idea is how to characterise the continuous dynamics of node representations, i.e. the derivatives of node representations, w.r.t. time. Inspired by existing diffusion-based methods on graphs (e.g. PageRank and epidemic models on social networks), we define the derivatives as a combination of the current node representations, the representations of neighbors, and the initial values of the nodes. We propose and analyse two possible dynamics on graphs---including each dimension of node representations (a.k.a. the feature channel) change independently or interact with each other---both with theoretical justification. The proposed continuous graph neural networks are robust to over-smoothing and hence allow us to build deeper networks, which in turn are able to capture the long-range dependencies between nodes. Experimental results on the task of node classification demonstrate the effectiveness of our proposed approach over competitive baselines.

研究动机与目标

• 激励图神经网络（GNNs）及其受限深度的过平滑问题。
• 通过受扩散方法启发的ODEs，在节点表示上引入连续时间动力学。
• 给出两种容量递增的ODEs：独立特征通道与通道交互。
• 从理论上分析固定点和长程依赖能力。
• 展示在节点分类上的实证收益，同时突出内存效率。

提出的方法

• 定义通过扩散启发传播推导的ODEs 使节点表示 H(t) 演化。
• 情形1：独立特征通道，导致 dH/dt = (A - I)H(t) + E，且 H(0)=E。
• 证明解析解并显示当 t→∞ 时收敛到 H(t) ≈ (I - A)^{-1}E。
• 情形2：引入通道交互，形式为 H'(t) = (A - I)H(t) + H(t)(W - I) + E，并使 W 可对角化且特征值 < 1。
• 使用泰勒近似简化对数项以实现实际计算。
• 采用伴随法实现对内存友好的反向传播。

实验结果

研究问题

• RQ1 Can continuous-time ODEs modeled on diffusion-inspired graph propagation avoid over-smoothing while enabling deep representations?
• RQ2 How do independent feature channels vs. interacting channels affect learning on graphs?
• RQ3 What are the theoretical properties (e.g., fixed points, convergence) of the proposed CGNN dynamics?
• RQ4 How does CGNN perform on standard node classification benchmarks compared to discrete GNNs and concurrent continuous approaches?

主要发现

• CGNN 在 Cora、Citeseer、Pubmed 和 NELL 上实现强劲的节点分类结果，优于若干基线。
• 模型对积分时间具有鲁棒性，随时间增长不再信息丢失，从而实现实质上无限深度的解释。
• 带通道交互（权重）的变体提供了可比的增益，在更具挑战性的图上可能具有优势。
• CGNN 通过伴随法实现对内存友好的反向传播，在深度上维持恒定的内存成本。
• CGNN 在若干数据集上优于离散传播基线（GCN、GAT）以及并行的基于ODE的方法（GODE）。
• 理论分析表明该动力学收敛到一个同时捕捉图结构与初始特征的静态表示。