[论文解读] DropEdge: Towards Deep Graph Convolutional Networks on Node Classification

Yu Rong, Wenbing Huang+2 more|arXiv (Cornell University)|Jul 25, 2019

Advanced Graph Neural Networks参考文献 33被引用 627

一句话总结

DropEdge在训练过程中随机丢弃边，以增强图数据并降低信息传递密度，从而实现更深的GCN并在多种骨架上提升节点分类性能。

ABSTRACT

\emph{Over-fitting} and \emph{over-smoothing} are two main obstacles of developing deep Graph Convolutional Networks (GCNs) for node classification. In particular, over-fitting weakens the generalization ability on small dataset, while over-smoothing impedes model training by isolating output representations from the input features with the increase in network depth. This paper proposes DropEdge, a novel and flexible technique to alleviate both issues. At its core, DropEdge randomly removes a certain number of edges from the input graph at each training epoch, acting like a data augmenter and also a message passing reducer. Furthermore, we theoretically demonstrate that DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by it. More importantly, our DropEdge is a general skill that can be equipped with many other backbone models (e.g. GCN, ResGCN, GraphSAGE, and JKNet) for enhanced performance. Extensive experiments on several benchmarks verify that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs. The effect of DropEdge on preventing over-smoothing is empirically visualized and validated as well. Codes are released on~\url{https://github.com/DropEdge/DropEdge}.

研究动机与目标

• 阐明深度GCN用于节点分类时过拟合和过平滑的挑战。
• 提出DropEdge作为一种灵活的边丢弃数据增强技术。
• 分析DropEdge如何减慢过平滑并在深层GCN中保留信息。

提出的方法

• DropEdge在每个训练轮次中随机移除输入图的一定比例p的边，以生成扰动的邻接矩阵。
• 按照Kipf & Welling的方法对被删除的邻接进行重新归一化，并在前向传播中使用它。
• 允许一种逐层变体，不同层使用独立的被删除邻接矩阵。
• 给出理论论证，表明DropEdge减慢过平滑的收敛或减少信息损失。
• 展示与GCN、ResGCN、JKNet、IncepGCN、GraphSAGE等骨架的兼容性及经验收益。

实验结果

研究问题

• RQ1 在小规模图上，DropEdge是否能缓解深度GCN的过拟合，同时保持表示质量？
• RQ2 DropEdge是否能够减慢过平滑，使深层GCN在不丢失输入特征信息的情况下变得更深？
• RQ3 DropEdge在不同骨架结构的节点分类任务中的迁移能力如何？
• RQ4 DropEdge对学习动态和各基准验证性能的实际影响如何？

主要发现

• DropEdge在多种数据集上对不同骨架的一致性提升测试准确率。
• DropEdge使得更深的架构（超过2层）表现更好，在某些设置甚至可避免内存问题。
• 该方法通过4层模型在Cora上显示更低的验证损失，减少过拟合。
• DropEdge可以与Dropout结合以获得额外收益。
• 逐层DropEdge在较高计算成本下提供了边际的训练改进。
• 在报道的实验中，DropEdge在Reddit等数据集上优于若干SOTA方法。