[论文解读] Residual Gated Graph ConvNets

Xavier Bresson, Thomas Laurent|arXiv (Cornell University)|Nov 20, 2017

Advanced Graph Neural Networks参考文献 25被引用 246

一句话总结

论文在变量长度图上严格比较了图RNNs和图卷积网络（graph ConvNets），提出了残余门控图卷积网络，并且表明卷积网络在准确性和速度方面优于RNNs，残差还带来额外提升。

ABSTRACT

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

研究动机与目标

• 动机：需要在跨领域（社交、脑科学、调控等）处理变长的图的神经网络。
• 比较图递归网络（RNNs）和图卷积网络（ConvNets），并将它们扩展到任意图大小。
• 设计并评估残余门控图卷积网络，以提升深度图模型的性能和训练。

提出的方法

• 回顾现有的图RNN和ConvNet架构。
• 提出多层图LSTM和带边门控的图卷积网络。
• 在层之间引入残差连接，以实现更深的架构。
• 在子图匹配和图聚类上进行解析控制的实验以比较模型。
• 采用批量归一化和标准优化时序，以确保公平比较。

实验结果

研究问题

• RQ1 哪种架构——图RNN还是图ConvNets——在处理具有任意大小和深度的图时更优？
• RQ2 门控边和残差连接是否提升图结构任务的学习？
• RQ3 在不同噪声和预算约束下，不同架构在子图匹配和图聚类上的表现如何？

主要发现

• 图卷积网络在所测试任务上比图RNNs的准确率高3-17%，训练速度快1.5-4倍。
• 图卷积网络在同一任务上比变分式（非学习）技术高36%。
• 最有效的图ConvNet使用门控边和残差连接。
• 残差性在堆叠超过6层时大约带来10%的性能提升。
• ConvNet 架构从增加深度中受益，而基于RNN的模型在层数过多时可能退化。