[论文解读] Can GCNs Go as Deep as CNNs

Guohao Li, Matthias Müller+2 more|arXiv (Cornell University)|Apr 7, 2019

Advanced Graph Neural Networks被引用 36

一句话总结

本文提出了一种受CNN中残差连接与密集连接启发的深度图卷积网络（GCN）框架，结合空洞卷积，实现了非常深的GCN训练——通过一个56层模型，在点云语义分割任务上相比最先进方法实现了3.7%的mIoU提升。

ABSTRACT

Convolutional Neural Networks (CNNs) achieve impressive performance in a wide variety of fields. Their success benefited from a massive boost when very deep CNN models were able to be reliably trained. Despite their merits, CNNs fail to properly address problems with non-Euclidean data. To overcome this challenge, Graph Convolutional Networks (GCNs) build graphs to represent non-Euclidean data, borrow concepts from CNNs, and apply them in training. GCNs show promising results, but they are usually limited to very shallow models due to the vanishing gradient problem. As a result, most state-of-the-art GCN models are no deeper than 3 or 4 layers. In this work, we present new ways to successfully train very deep GCNs. We do this by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures. Extensive experiments show the positive effect of these deep GCN frameworks. Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3.7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation. We believe that the community can greatly benefit from this work, as it opens up many opportunities for advancing GCN-based research.

研究动机与目标

• 解决现有模型中限制GCN深度的梯度消失问题。
• 实现可训练的极深GCN架构，与深度CNN相媲美。
• 提升在非欧几里得数据上的性能，特别是在点云语义分割任务中。
• 将CNN中成功的技术——残差连接与密集连接、空洞卷积——适配到GCN框架中。

提出的方法

• 将CNN中的残差连接与密集跳跃连接适配到GCN中，以缓解深层架构中的梯度消失问题。
• 在GCN中引入空洞卷积，以扩展感受野，同时不增加参数量或层数。
• 设计一种使用残差连接的深层GCN架构，支持训练至56层。
• 采用具有可学习滤波器的图卷积层，聚合图中邻近节点的特征。
• 使用恒等捷径以保持梯度在各层间的流动，提升训练稳定性。
• 通过空洞卷积实现多尺度感受野，以增强深层GCN中的特征表示能力。

实验结果

研究问题

• RQ1 GCN能否像CNN一样训练到极深，而不会出现梯度消失？
• RQ2 残差连接与密集连接如何改善GCN架构中的训练深度？
• RQ3 空洞卷积对GCN中感受野扩展有何影响？
• RQ4 56层GCN能否在点云语义分割任务上达到最先进性能？
• RQ5 在GCN中结合残差、密集与空洞机制能带来多大的性能增益？

主要发现

• 所提出的深度GCN框架成功训练了56层模型，远超典型GCN通常限制在3–4层的深度。
• 56层GCN在点云语义分割任务上相比最先进方法实现了3.7%的mIoU提升。
• 残差连接与密集连接有效缓解了深层GCN中的梯度消失问题。
• 空洞卷积通过扩展感受野增强了特征表示，且无需增加模型深度。
• 残差、密集与空洞机制的结合在实验中带来了持续的性能增益。
• 该框架表明，GCN在非欧几里得数据任务中可实现与深度CNN相当的性能。