[Paper Review] Homogeneous Network Embedding for Massive Graphs via Reweighted Personalized PageRank

Renchi Yang, J. Y. Shi+3 more|arXiv (Cornell University)|Jun 17, 2019

Advanced Graph Neural Networks71 references4 citations

TL;DR

This paper proposes Node-Reweighted PageRank (NRP), a scalable and effective method for homogeneous network embedding in massive graphs using reweighted personalized PageRank. By integrating node degree information into PPR to correct its local bias, NRP achieves state-of-the-art performance in link prediction, node classification, and graph reconstruction while running in under 4 hours on a billion-edge Twitter graph using a single CPU core.

ABSTRACT

Given an input graph G and a node v in G, homogeneous network embedding (HNE) maps the graph structure in the vicinity of v to a compact, fixed-dimensional feature vector. This paper focuses on HNE for massive graphs, e.g., with billions of edges. On this scale, most existing approaches fail, as they incur either prohibitively high costs, or severely compromised result utility. Our proposed solution, called Node-Reweighted PageRank (NRP), is based on a classic idea of deriving embedding vectors from pairwise personalized PageRank (PPR) values. Our contributions are twofold: first, we design a simple and efficient baseline HNE method based on PPR that is capable of handling billion-edge graphs on commodity hardware; second and more importantly, we identify an inherent drawback of vanilla PPR, and address it in our main proposal NRP. Specifically, PPR was designed for a very different purpose, i.e., ranking nodes in G based on their relative importance from a source node's perspective. In contrast, HNE aims to build node embeddings considering the whole graph. Consequently, node embeddings derived directly from PPR are of suboptimal utility. The proposed NRP approach overcomes the above deficiency through an effective and efficient node reweighting algorithm, which augments PPR values with node degree information, and iteratively adjusts embedding vectors accordingly. Overall, NRP takes O(mlogn) time and O(m) space to compute all node embeddings for a graph with m edges and n nodes. Our extensive experiments that compare NRP against 18 existing solutions over 7 real graphs demonstrate that NRP achieves higher result utility than all the solutions for link prediction, graph reconstruction and node classification, while being up to orders of magnitude faster. In particular, on a billion-edge Twitter graph, NRP terminates within 4 hours, using a single CPU core.

Motivation & Objective

• Address the scalability and utility limitations of existing homogeneous network embedding (HNE) methods on massive graphs with billions of edges.
• Identify the inherent flaw in vanilla personalized PageRank (PPR) for HNE: PPR is designed for local ranking, not global graph representation, leading to suboptimal embeddings.
• Develop a node reweighting mechanism that integrates node degree information into PPR to improve embedding quality while maintaining efficiency.
• Achieve high performance in downstream tasks (link prediction, node classification, graph reconstruction) on large-scale graphs using commodity hardware.

Proposed method

• Propose Node-Reweighted PageRank (NRP), a novel HNE method that enhances standard PPR by incorporating node degree information through a reweighting strategy.
• Design an iterative algorithm that adjusts embedding vectors based on reweighted PPR values, using a closed-form update rule derived from minimizing a regularized objective function.
• Use a forward and backward weight update mechanism (Algorithms 2 and 4) to efficiently compute embeddings in O(m log n) time and O(m) space, where m is the number of edges.
• Introduce a parameterized approximation scheme with parameters ℓ₁, ℓ₂, α, and ε to control accuracy and convergence, enabling trade-offs between speed and precision.
• Leverage the fact that PPR values summarize infinite random walks, using them as a compact, informative proxy for multi-hop topological structure.
• Apply a symmetric reweighting function that scales PPR values by node degrees, ensuring high-degree nodes contribute meaningfully to the embedding without distorting the global structure.

Experimental results

Research questions

• RQ1 Can personalized PageRank be effectively repurposed for scalable homogeneous network embedding in massive graphs without incurring prohibitive computational costs?
• RQ2 What are the key limitations of using vanilla PPR for HNE, and how do they affect embedding quality in global graph representation tasks?
• RQ3 Can node degree information be integrated into PPR to correct its local bias and improve embedding utility for downstream machine learning tasks?
• RQ4 Is it possible to design an efficient, iterative reweighting algorithm that maintains linear time complexity while significantly improving embedding quality on large graphs?

Key findings

• NRP achieves higher result utility than all 18 compared methods in link prediction, graph reconstruction, and node classification across 7 real-world graphs.
• On the billion-edge Twitter graph, NRP computes all node embeddings in under 4 hours using only a single CPU core, demonstrating exceptional scalability.
• NRP outperforms PPR-based baselines like STRAP, VERSE, and APP by at least 0.7% in AUC on the directed Digg dataset, showing superior performance on evolving graphs.
• The running time of NRP grows linearly with both the number of nodes and edges, confirming its O(m log n) time complexity and strong scalability.
• Increasing ℓ₂ and ℓ₁ parameters increases running time, while α has minimal impact, aligning with the theoretical time complexity model.
• The node reweighting mechanism significantly improves embedding quality by correcting the local bias of PPR, as validated across multiple benchmark tasks and datasets.