[论文解读] Graph Pointer Neural Networks

Tianmeng Yang, Yujing Wang+4 more|arXiv (Cornell University)|Oct 3, 2021

Advanced Graph Neural Networks被引用 4

一句话总结

该论文提出图指针神经网络（GPNN），一种新颖的图神经网络（GNN）框架，通过指针网络有选择性地识别并排序每个中心节点的最相关节点（包括局部和远程节点），实现非局部、抗噪声的特征聚合。通过与端到端训练联合优化指针网络，GPNN显著减少了过平滑问题，并在异配性图上提升了节点分类准确率，相较于最先进方法平均提升6.3%。

ABSTRACT

Graph Neural Networks (GNNs) have shown advantages in various graph-based applications. Most existing GNNs assume strong homophily of graph structure and apply permutation-invariant local aggregation of neighbors to learn a representation for each node. However, they fail to generalize to heterophilic graphs, where most neighboring nodes have different labels or features, and the relevant nodes are distant. Few recent studies attempt to address this problem by combining multiple hops of hidden representations of central nodes (i.e., multi-hop-based approaches) or sorting the neighboring nodes based on attention scores (i.e., ranking-based approaches). As a result, these approaches have some apparent limitations. On the one hand, multi-hop-based approaches do not explicitly distinguish relevant nodes from a large number of multi-hop neighborhoods, leading to a severe over-smoothing problem. On the other hand, ranking-based models do not joint-optimize node ranking with end tasks and result in sub-optimal solutions. In this work, we present Graph Pointer Neural Networks (GPNN) to tackle the challenges mentioned above. We leverage a pointer network to select the most relevant nodes from a large amount of multi-hop neighborhoods, which constructs an ordered sequence according to the relationship with the central node. 1D convolution is then applied to extract high-level features from the node sequence. The pointer-network-based ranker in GPNN is joint-optimized with other parts in an end-to-end manner. Extensive experiments are conducted on six public node classification datasets with heterophilic graphs. The results show that GPNN significantly improves the classification performance of state-of-the-art methods. In addition, analyses also reveal the privilege of the proposed GPNN in filtering out irrelevant neighbors and reducing over-smoothing.

研究动机与目标

• 解决现有GNN在异配性图中的局限性，其中相连节点的特征或标签往往不相似。
• 克服深层GNN中因无差别邻居聚合导致的过平滑问题。
• 通过实现远距离但语义相关的节点的非局部、选择性聚合，改进节点表征学习。
• 与下游任务联合优化节点排序，避免因不可微排序过程导致的次优解。
• 开发一种可扩展且高效的方法框架，在真实世界异配性图数据集上优于现有方法。

提出的方法

• 从每个中心节点的局部及多跳邻域中采样节点序列。
• 使用指针网络根据其与中心节点的相关性对采样节点进行排序，生成最具信息量邻居的有序序列。
• 对排序后的序列应用一维卷积层，以提取高层次的结构与语义特征。
• 将指针网络作为可微分、端到端可训练模块，与GNN架构其余部分联合优化。
• 将中心节点的自指特征与排序邻居序列的上下文特征相结合，生成最终的节点表征。
• 采用可微分排序机制，使反向传播能够同时优化节点选择与表征学习。

实验结果

研究问题

• RQ1 可微分、端到端可训练的指针网络是否能有效识别异配性图中即使距离较远的最相关邻居？
• RQ2 与标准GNN在深层堆叠结构下的聚合方式相比，使用指针网络进行选择性非局部聚合是否能减少过平滑？
• RQ3 GPNN所选择邻居的同配性比率与非可微排序方法（如Node2Seq）相比如何？
• RQ4 相较于最先进GNN模型，GPNN在真实世界异配性图基准测试中节点分类准确率的提升程度如何？
• RQ5 节点排序与表征学习的联合优化是否能带来优于独立或启发式排序策略的性能表现？

主要发现

• 在六个公开的异配性图数据集上，GPNN相较于最佳现有最先进方法，平均节点分类准确率提升6.3%。
• 在Chameleon数据集上，GPNN性能较第二好的方法提升1.9%；在Cornell数据集上提升3.0%，表明在多样化基准上具有持续的性能增益。
• GPNN所选择的五个最相关邻居的同配性比率比Node2Seq基于注意力的方法所选邻居高出46%，表明其具有更优的邻居过滤能力。
• GPNN表现出显著减少的过平滑现象：当层数从2层增至5层时，测试准确率仅下降6%，而GCN和GAT在Chameleon数据集上的下降幅度分别为18%和33%。
• 指针网络与GNN的联合优化优于非可微排序方法，表现为更高的同配性比率与准确率增益。
• GPNN始终优于H2GCN与Node2Seq，后者未显式区分相关与无关的多跳邻居，证实了选择性、可微邻居选择的优势。