[论文解读] Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks

Jianxin Li, Hao Peng+5 more|arXiv (Cornell University)|Apr 16, 2021

Advanced Graph Neural Networks参考文献 52被引用 16

一句话总结

该论文提出了一种新型框架——高阶属性增强图神经网络（HAE GNN），通过联合利用元路径和元图来捕捉丰富的异构语义信息，同时采用自注意力机制建模一阶及更高阶邻居之间的内容相关交互。HAE GNN 在节点分类与聚类任务中实现了最先进性能，相较于现有方法，其内存效率和可解释性均得到提升。

ABSTRACT

Graph neural networks (GNNs) have been widely used in deep learning on graphs. They can learn effective node representations that achieve superior performances in graph analysis tasks such as node classification and node clustering. However, most methods ignore the heterogeneity in real-world graphs. Methods designed for heterogeneous graphs, on the other hand, fail to learn complex semantic representations because they only use meta-paths instead of meta-graphs. Furthermore, they cannot fully capture the content-based correlations between nodes, as they either do not use the self-attention mechanism or only use it to consider the immediate neighbors of each node, ignoring the higher-order neighbors. We propose a novel Higher-order Attribute-Enhancing (HAE) framework that enhances node embedding in a layer-by-layer manner. Under the HAE framework, we propose a Higher-order Attribute-Enhancing Graph Neural Network (HAEGNN) for heterogeneous network representation learning. HAEGNN simultaneously incorporates meta-paths and meta-graphs for rich, heterogeneous semantics, and leverages the self-attention mechanism to explore content-based nodes interactions. The unique higher-order architecture of HAEGNN allows examining the first-order as well as higher-order neighborhoods. Moreover, HAEGNN shows good explainability as it learns the importances of different meta-paths and meta-graphs. HAEGNN is also memory-efficient, for it avoids per meta-path based matrix calculation. Experimental results not only show HAEGNN superior performance against the state-of-the-art methods in node classification, node clustering, and visualization, but also demonstrate its superiorities in terms of memory efficiency and explainability.

研究动机与目标

• 解决现有异构图神经网络仅依赖元路径而忽略高阶邻域交互的局限性。
• 通过整合元路径与元图，实现更丰富、更复杂的语义表示学习。
• 利用自注意力机制在多阶邻居之间建模基于内容的节点交互，超越直接邻居的范围。
• 通过学习不同元路径与元图的重要性，提升模型的可解释性。
• 通过避免对每条元路径进行矩阵运算，实现内存效率优化，提升计算效率。

提出的方法

• HAE GNN 采用逐层架构，通过高阶消息传递机制捕捉来自一阶及更高阶邻域的节点表征。
• 在模式层面上整合元路径与元图，以捕捉异构信息网络（HINs）中节点之间的多样化语义关系。
• 在节点及其高阶邻居之间应用自注意力机制，以建模基于内容的依赖关系，并动态加权重要邻居。
• 模型学习不同元路径与元图的注意力权重，从而通过识别对节点表征贡献最大的语义路径，实现可解释性。
• 通过避免对每条元路径执行矩阵运算，采用统一且参数高效的注意力机制，实现内存效率优化。
• 该框架支持归纳学习与直推学习，实验验证基于 DBLP、IMDB 和 HUAWEI 数据集。

Figure 1: The architecture of the proposed Higher-order Attribute-Enhancing Graph Neural Network (HAE GNN ). HAE GNN contains three components, i.e., (a) Semantic-Based Convolutional Layer (SCL), (b) Content-Based Self-Attention Layer (CAL), and (c) Higher-Order Attribute-Enhancing Framework (HAE).

实验结果

研究问题

• RQ1 与仅使用元路径相比，结合元路径与元图是否能提升异构网络中的语义表征学习？
• RQ2 对高阶邻居之间交互的建模是否能带来更优的节点表征与下游任务性能？
• RQ3 自注意力机制是否能有效捕捉节点与其高阶邻居之间的基于内容的依赖关系？
• RQ4 所提出的 HAE GNN 模型在性能、内存效率与可解释性方面相较于最先进异构图神经网络表现如何？
• RQ5 为在不过拟合的前提下最大化性能，邻域聚合的最佳阶数是多少？

主要发现

• HAE GNN 在节点分类任务中实现了最先进性能，其中 HAE GNN^4l 在 DBLP 上相较于 HAE GNN^2l 取得了更高的 Macro-F1 与 Micro-F1 分数。
• 在 DBLP 数据集上，HAE GNN^4l 表现优于 HAE GNN^2l，表明高阶建模在复杂、基于引用的网络中尤为有效。
• HAE GNN^2l 的内存消耗比 HAN 减少 19.6%，比 GTN 减少 7.4%，展现出更优的内存效率。
• HAE GNN^2l 在 20 秒内完成 100 个训练周期，优于 GTN 的 23 秒，尽管使用了多头注意力机制，仍表现出更快的训练速度。
• HAE GNN 模型通过学习并排序不同元路径与元图的重要性，实现了改进的可解释性。
• 当阶数超过 4 后性能开始下降，表明模型深度与泛化能力之间存在权衡。

Figure 2: Meta-schema, meta-paths and meta-graphs of the DBLP, IMDB and HUAWEI datasets. In (a), A, P, C, and T stand for Author , Paper , Conference , and Term , respectively. In (b), A, M, D, and R stand for Actor , Movie , Director , and Rating , respectively. In (c), U, A, and T stand for User ,