[论文解读] Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning

Xiao Wang, Nian Liu+2 more|arXiv (Cornell University)|May 19, 2021

Advanced Graph Neural Networks参考文献 34被引用 27

一句话总结

引入HeCo，一种自监督的异质图神经网络，利用跨视图对比学习在网络模式与元路径视图之间进行，具备视图掩码和难负样本扩展，在多个数据集上优于基线方法。

ABSTRACT

Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use in reality since labels are usually scarce in real applications. Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo. Different from traditional contrastive learning which only focuses on contrasting positive and negative samples, HeCo employs cross-viewcontrastive mechanism. Specifically, two views of a HIN (network schema and meta-path views) are proposed to learn node embeddings, so as to capture both of local and high-order structures simultaneously. Then the cross-view contrastive learning, as well as a view mask mechanism, is proposed, which is able to extract the positive and negative embeddings from two views. This enables the two views to collaboratively supervise each other and finally learn high-level node embeddings. Moreover, two extensions of HeCo are designed to generate harder negative samples with high quality, which further boosts the performance of HeCo. Extensive experiments conducted on a variety of real-world networks show the superior performance of the proposed methods over the state-of-the-arts.

研究动机与目标

• 解决异质信息网络（HIN）中的标签稀缺问题，通过为HGNNs实现自监督学习。
• 设计跨视图对比机制，以捕捉HINs中的局部（网络模式）和高阶（元路径）结构。
• 引入视图掩码，增强视图多样性与视图之间的协同监督。
• 提供扩展以生成更高质量的负样本，提升对比学习性能。

提出的方法

• 定义HIN的两种视图：网络模式视图和元路径视图，并在每个视图中学习节点嵌入。
• 将节点特征投影到具有类型特异性的线性映射的共同潜在空间。
• 网络模式视图编码器通过节点级和类型级注意力结合邻居采样来聚合多类型邻居，以控制多样性。
• 元路径视图编码器应用元路径特定的GCN和语义级注意力来融合多个元路径嵌入。
• 应用跨视图对比损失，将一个视图的嵌入作为锚点，另一视图的嵌入作为正/负样本，采用多正对比目标并共享的MLP投影。
• 引入视图掩码机制，通过在编码时遮蔽每个视图的部分内容来增加跨视图多样性，强制获得互补监督。
• 提出两种扩展，HeCo_GAN和HeCo_MU，用以生成更高质量的负样本（基于GAN的负样本和受MixUp启发的负样本），并提升训练效率。

实验结果

研究问题

• RQ1 如何在没有标签的情况下有效地将自监督学习应用于异质信息网络（HINs）？
• RQ2 跨网络模式和元路径视图的跨视图对比学习是否能比单视图方法更好地捕捉局部与高阶的HIN结构？
• RQ3 哪些机制（例如视图掩码、难负样本）能提升HGNN中对比信号的质量和有用性？
• RQ4 跨视图对比方法在真实世界HIN数据集上是否优于现有的无监督和半监督HGNN？

主要发现

• HeCo在多个人真实世界HIN数据集的节点分类任务中，一致优于最先进的无监督以及部分半监督基线。
• 跨视图学习（网络模式 vs. 元路径）产生比单视图方法更具辨别力的节点嵌入，并有助于缓解单一视图信号中的噪声。
• 视图掩码提高视图多样性，强化跨视图监督，带来更好的嵌入。
• 扩展HeCo_GAN和HeCo_MU提供更高质量的负样本，进一步提升性能。
• 所提出的跨视图对比框架证明了自监督学习在异质图中的有效性，在某些场景下甚至可超越依赖标签的方法。