[论文解读] Improving Graph Neural Networks with Simple Architecture Design

Sunil Kumar Maurya, Xin Liu+1 more|arXiv (Cornell University)|May 17, 2021

Advanced Graph Neural Networks参考文献 36被引用 17

一句话总结

该论文提出FSGNN，一种简单而高效的图神经网络，通过基于softmax的软选择与跳数归一化，将特征聚合与表征学习解耦，在节点分类任务中实现比最先进模型高64%的准确率，同时支持可解释的特征重要性分析，并可扩展至百亿条边的图。

ABSTRACT

Graph Neural Networks have emerged as a useful tool to learn on the data by applying additional constraints based on the graph structure. These graphs are often created with assumed intrinsic relations between the entities. In recent years, there have been tremendous improvements in the architecture design, pushing the performance up in various prediction tasks. In general, these neural architectures combine layer depth and node feature aggregation steps. This makes it challenging to analyze the importance of features at various hops and the expressiveness of the neural network layers. As different graph datasets show varying levels of homophily and heterophily in features and class label distribution, it becomes essential to understand which features are important for the prediction tasks without any prior information. In this work, we decouple the node feature aggregation step and depth of graph neural network and introduce several key design strategies for graph neural networks. More specifically, we propose to use softmax as a regularizer and "Soft-Selector" of features aggregated from neighbors at different hop distances; and "Hop-Normalization" over GNN layers. Combining these techniques, we present a simple and shallow model, Feature Selection Graph Neural Network (FSGNN), and show empirically that the proposed model outperforms other state of the art GNN models and achieves up to 64% improvements in accuracy on node classification tasks. Moreover, analyzing the learned soft-selection parameters of the model provides a simple way to study the importance of features in the prediction tasks. Finally, we demonstrate with experiments that the model is scalable for large graphs with millions of nodes and billions of edges.

研究动机与目标

• 为解决堆叠GNN层中特征重要性与模型表达能力难以解耦的挑战。
• 在不依赖深层架构的前提下，提升同质图与异质图数据集上的性能。
• 通过可学习的软选择参数，实现可解释的特征重要性分析。
• 设计一种可扩展的浅层GNN模型，适用于包含数百万个节点和数十亿条边的大规模图。
• 证明简单的架构修改可超越复杂且深层的GNN变体。

提出的方法

• 通过将邻居特征聚合与最终MLP变换分离，实现特征聚合与表征学习的解耦。
• 引入基于softmax的软选择器，动态加权来自不同跳数距离的聚合特征。
• 应用跳数归一化以稳定训练过程，防止梯度消失或爆炸。
• 采用两层架构，包含可学习的标量权重与ReLU激活函数，避免使用残差连接或复杂归一化方法。
• 采用独特的特征映射策略，使模型能够学习来自不同跳数的哪些特征最为相关。
• 使用Adam优化器进行训练，并通过网格搜索调整超参数，包括初始学习率、权重衰减和dropout率。

实验结果

研究问题

• RQ1 是否一种简单且浅层的GNN架构能在节点分类任务中超越更深、更复杂的模型？
• RQ2 如何有效建模并解释多跳距离下的特征重要性？
• RQ3 将特征聚合与表征学习解耦是否能提升模型性能与泛化能力？
• RQ4 所提模型是否能在同质图与异质图数据集上均保持高性能？
• RQ5 该模型是否可扩展至超大规模图，如包含超过十亿条边的ogbn-papers100M？

主要发现

• 在Cora数据集上，FSGNN实现67.17%的测试准确率，优于SGC（63.29%）、Node2Vec（58.07%）和SIGN（65.11%），较之前最先进模型提升64%。
• 在不同跳数（3、8、16、32）下，模型性能保持一致，表明由于有效的软选择机制，其对聚合深度具有鲁棒性。
• 在ogbn-papers100M数据集上，FSGNN的准确率高于所有先前的最先进模型，证明其在大规模图上的可扩展性。
• 对学习到的软选择权重分析显示，Chameleon与Squirrel数据集表现出强烈的异质性，与先前认为其异质性质量较低的假设相反。
• 随着跳数增加，模型性能下降极小，证实软注意力机制能有效抑制噪声或无关的长距离特征。
• 消融实验表明，软选择与跳数归一化均为关键组件，其移除将导致性能显著下降。