[论文解读] Scalable Graph Embeddings via Sparse Transpose Proximities

Yuan Yin, Zhewei Wei|arXiv (Cornell University)|May 16, 2019

Advanced Graph Neural Networks参考文献 36被引用 4

一句话总结

该论文提出STRAP，一种可扩展的图嵌入方法，利用稀疏转置邻近度来保留有向图中的入度和出度分布，并消除无向图中随机游走方法的冲突优化目标。通过利用反向传播算法高效计算稀疏个性化PageRank作为转置邻近度，并应用稀疏性约束的非线性分解，STRAP在链接预测、图重构和节点分类任务中均达到最先进性能，同时保持与图规模线性可扩展性。

ABSTRACT

Graph embedding learns low-dimensional representations for nodes in a graph and effectively preserves the graph structure. Recently, a significant amount of progress has been made toward this emerging research area. However, there are several fundamental problems that remain open. First, existing methods fail to preserve the out-degree distributions on directed graphs. Second, many existing methods employ random walk based proximities and thus suffer from conflicting optimization goals on undirected graphs. Finally, existing factorization methods are unable to achieve scalability and non-linearity simultaneously. This paper presents an in-depth study on graph embedding techniques on both directed and undirected graphs. We analyze the fundamental reasons that lead to the distortion of out-degree distributions and to the conflicting optimization goals. We propose {\em transpose proximity}, a unified approach that solves both problems. Based on the concept of transpose proximity, we design \strap, a factorization based graph embedding algorithm that achieves scalability and non-linearity simultaneously. \strap makes use of the {\em backward push} algorithm to efficiently compute the sparse {\em Personalized PageRank (PPR)} as its transpose proximities. By imposing the sparsity constraint, we are able to apply non-linear operations to the proximity matrix and perform efficient matrix factorization to derive the embedding vectors. Finally, we present an extensive experimental study that evaluates the effectiveness of various graph embedding algorithms, and we show that \strap outperforms the state-of-the-art methods in terms of effectiveness and scalability.

研究动机与目标

• 解决现有图嵌入方法在有向图上无法保留出度分布的问题。
• 解决无向图中基于随机游走方法的冲突优化目标问题。
• 在基于因子分解的图嵌入中实现可扩展性与非线性。
• 通过一种新型邻近度度量，统一保留入度与出度分布并实现无冲突优化。

提出的方法

• 引入转置邻近度作为统一度量，通过结合前向和后向PPR捕捉入度与出度信息。
• 使用反向传播算法高效计算大规模图的稀疏个性化PageRank（PPR）。
• 对邻近度矩阵施加稀疏性约束，以在矩阵分解前实现高效的非线性操作。
• 对非线性变换后的邻近度矩阵执行低秩矩阵分解，以生成节点嵌入。
• 利用PPR的稀疏性，确保与边数呈线性时间复杂度。
• 设计STRAP为一种基于因子分解的算法，集成转置邻近度、可扩展计算与非线性建模。

实验结果

研究问题

• RQ1 统一的邻近度度量能否在有向图中同时保留入度与出度分布？
• RQ2 转置邻近度是否能消除无向图中冲突的优化目标？
• RQ3 基于因子分解的方法能否在大规模图嵌入中同时实现可扩展性与非线性？
• RQ4 衰减因子α和误差容限ε的选择如何影响精度与运行时间之间的权衡？

主要发现

• STRAP在链接预测与图重构任务中优于最先进方法，在WikiVote数据集上精度最高达27.97%。
• STRAP在节点结构角色分类任务中表现最佳，在巴西数据集上达到70.71%准确率，在欧洲数据集上达到65.50%（在90%数据采样下）。
• STRAP在最大图（BC）上仅需39秒运行时间，显著快于DeepWalk（1200秒）与Node2Vec（280秒），展现出卓越的可扩展性。
• STRAP的运行时间与边数呈线性增长，证实了其理论效率。
• 当α从1降至0时，方法从归纳性转向归纳性更强的特性，显示出可调节的归纳偏差。
• 降低ε可提升重构精度，但会增加运行时间，证实了有效权衡控制。