[Paper Review] Scalable Graph Embeddings via Sparse Transpose Proximities

Yuan Yin, Zhewei Wei|arXiv (Cornell University)|May 16, 2019

Advanced Graph Neural Networks36 references4 citations

TL;DR

This paper proposes STRAP, a scalable graph embedding method that uses sparse transpose proximities to preserve both in- and out-degree distributions in directed graphs and eliminate conflicting optimization goals in undirected graphs. By leveraging backward push to compute sparse Personalized PageRank as transpose proximities and applying sparsity-constrained non-linear factorization, STRAP achieves state-of-the-art performance in link prediction, graph reconstruction, and node classification while maintaining linear scalability with respect to graph size.

ABSTRACT

Graph embedding learns low-dimensional representations for nodes in a graph and effectively preserves the graph structure. Recently, a significant amount of progress has been made toward this emerging research area. However, there are several fundamental problems that remain open. First, existing methods fail to preserve the out-degree distributions on directed graphs. Second, many existing methods employ random walk based proximities and thus suffer from conflicting optimization goals on undirected graphs. Finally, existing factorization methods are unable to achieve scalability and non-linearity simultaneously. This paper presents an in-depth study on graph embedding techniques on both directed and undirected graphs. We analyze the fundamental reasons that lead to the distortion of out-degree distributions and to the conflicting optimization goals. We propose {\em transpose proximity}, a unified approach that solves both problems. Based on the concept of transpose proximity, we design \strap, a factorization based graph embedding algorithm that achieves scalability and non-linearity simultaneously. \strap makes use of the {\em backward push} algorithm to efficiently compute the sparse {\em Personalized PageRank (PPR)} as its transpose proximities. By imposing the sparsity constraint, we are able to apply non-linear operations to the proximity matrix and perform efficient matrix factorization to derive the embedding vectors. Finally, we present an extensive experimental study that evaluates the effectiveness of various graph embedding algorithms, and we show that \strap outperforms the state-of-the-art methods in terms of effectiveness and scalability.

Motivation & Objective

• To address the failure of existing graph embedding methods in preserving out-degree distributions on directed graphs.
• To resolve conflicting optimization goals in random walk-based methods on undirected graphs.
• To achieve both scalability and non-linearity in factorization-based graph embedding.
• To unify in- and out-degree preservation and conflict-free optimization through a novel proximity measure.

Proposed method

• Introduce transpose proximity as a unified measure that captures both in- and out-degree information by combining forward and backward PPR.
• Use the backward push algorithm to efficiently compute sparse Personalized PageRank (PPR) for large-scale graphs.
• Apply sparsity constraints to the proximity matrix to enable efficient non-linear operations before matrix factorization.
• Perform low-rank matrix factorization on the non-linearly transformed proximity matrix to derive node embeddings.
• Leverage the sparsity of PPR to ensure linear time complexity with respect to the number of edges.
• Design STRAP as a factorization-based algorithm that integrates transpose proximity with scalable computation and non-linear modeling.

Experimental results

Research questions

• RQ1 Can a unified proximity measure preserve both in- and out-degree distributions in directed graphs?
• RQ2 Does transpose proximity eliminate conflicting optimization objectives in undirected graphs?
• RQ3 Can a factorization-based method achieve both scalability and non-linearity in large-scale graph embedding?
• RQ4 How does the choice of decay factor α and error tolerance ε affect the trade-off between precision and running time?

Key findings

• STRAP outperforms state-of-the-art methods in link prediction and graph reconstruction, achieving up to 27.97% precision on WikiVote.
• STRAP achieves the best performance in node structural role classification, reaching 70.71% accuracy on Brazil and 65.50% on Euro at 90% data sampling.
• STRAP runs in 39 seconds on the largest graph (BC), significantly faster than DeepWalk (1200 seconds) and Node2Vec (280 seconds), demonstrating superior scalability.
• The running time of STRAP scales linearly with the number of edges, confirming theoretical efficiency.
• As α decreases from 1 to 0, the method shifts from inductive to transductive strength, showing tunable inductive bias.
• Reducing ε improves reconstruction precision at the cost of increased running time, confirming effective trade-off control.