[论文解读] Link Prediction with Non-Contrastive Learning

William Shiao, Zhichun Guo+4 more|arXiv (Cornell University)|Nov 25, 2022

Advanced Graph Neural Networks被引用 5

一句话总结

该论文提出T-BGRL，一种新颖的非对比图自监督学习框架，用于链接预测。通过应用低成本的输入扰动而非昂贵的负采样，该方法在归纳设置中提升了泛化能力。相较于基线非对比方法，其在Hits@50上最高提升达120%，且比最佳对比基线快14倍，同时在6个数据集中的5个上保持了强劲性能。

ABSTRACT

A recent focal area in the space of graph neural networks (GNNs) is graph self-supervised learning (SSL), which aims to derive useful node representations without labeled data. Notably, many state-of-the-art graph SSL methods are contrastive methods, which use a combination of positive and negative samples to learn node representations. Owing to challenges in negative sampling (slowness and model sensitivity), recent literature introduced non-contrastive methods, which instead only use positive samples. Though such methods have shown promising performance in node-level tasks, their suitability for link prediction tasks, which are concerned with predicting link existence between pairs of nodes (and have broad applicability to recommendation systems contexts) is yet unexplored. In this work, we extensively evaluate the performance of existing non-contrastive methods for link prediction in both transductive and inductive settings. While most existing non-contrastive methods perform poorly overall, we find that, surprisingly, BGRL generally performs well in transductive settings. However, it performs poorly in the more realistic inductive settings where the model has to generalize to links to/from unseen nodes. We find that non-contrastive models tend to overfit to the training graph and use this analysis to propose T-BGRL, a novel non-contrastive framework that incorporates cheap corruptions to improve the generalization ability of the model. This simple modification strongly improves inductive performance in 5/6 of our datasets, with up to a 120% improvement in Hits@50--all with comparable speed to other non-contrastive baselines and up to 14x faster than the best-performing contrastive baseline. Our work imparts interesting findings about non-contrastive learning for link prediction and paves the way for future researchers to further expand upon this area.

研究动机与目标

• 评估非对比自监督学习方法在推荐系统中关键任务——链接预测中的适用性。
• 识别现有非对比方法（尤其是BGRL）在归纳链接预测设置中表现不佳的原因。
• 通过引入轻量级基于扰动的正则化机制，解决非对比模型在未见节点上过拟合的问题。
• 开发一种方法，在保持非对比学习速度优势的同时，实现与对比基线相当或更优的归纳链接预测性能。

提出的方法

• T-BGRL通过引入一种扰动函数扩展了BGRL，该函数在无需显式负采样的情况下生成输入图的低成本伪负视图。
• 扰动过程对原始图输入应用简单、低成本的变换（例如节点特征掩码、边删除）以生成增强视图。
• 模型在原始视图与扰动视图之间使用类似对比的损失函数，但无需负样本，从而保持了非对比学习的效率。
• 损失函数鼓励原始视图与扰动视图之间的特征不变性，从而提升对未见节点的鲁棒性与泛化能力。
• 该框架使用标准GNN编码器和预测头端到端训练，扰动在前向传播过程中应用。
• 该方法在六个基准数据集上评估了归纳与归纳链接预测设置下的表现。

实验结果

研究问题

• RQ1 为何非对比图SSL方法（包括BGRL）在归纳链接预测中表现不佳，尽管其在归纳设置中表现强劲？
• RQ2 在链接预测中，当模型泛化到未见节点时，为何会出现观察到的过拟合现象？
• RQ3 轻量级扰动机制是否能在不重新引入负采样计算成本的前提下，提升非对比链接预测的泛化能力？
• RQ4 T-BGRL在归纳链接预测准确率和训练速度方面，与非对比及对比基线相比表现如何？

主要发现

• BGRL在归纳链接预测中表现良好，在6个数据集中的4个上与对比模型（如margin-loss GCN）性能相当或更优。
• 在归纳链接预测中，BGRL因过拟合导致性能显著下降，尤其体现在将涉及未见节点的正负节点对表示拉远方面。
• T-BGRL在6个数据集中的5个上提升了BGRL的归纳性能，Hits@50最高提升达120%。
• T-BGRL的训练速度与其它非对比方法相当，且在Coauthor-Phy数据集上比最佳对比基线（margin-loss GCN）快14倍。
• T-BGRL中的基于扰动的增强方法有效降低了过拟合，增强了泛化能力，且无需昂贵的负采样。
• 所提方法在非对比方法中达到最先进性能，并在归纳链接预测中匹配或超越对比基线，同时显著更快。