[Paper Review] Initialization for Network Embedding: A Graph Partition Approach

Wenqing Lin, Feng He+3 more|arXiv (Cornell University)|Aug 28, 2019

Advanced Graph Neural Networks30 references4 citations

TL;DR

This paper proposes GPA, a graph partition-based initialization method for network embedding that improves both effectiveness and efficiency. By partitioning the input graph into disjoint subgraphs, constructing a smaller abstract graph, computing its embedding via a modified skip-gram model, and propagating it back to the original nodes, GPA boosts performance by up to 8.74% on node classification and reduces training time by at least 20% across datasets.

ABSTRACT

Network embedding has been intensively studied in the literature and widely used in various applications, such as link prediction and node classification. While previous work focus on the design of new algorithms or are tailored for various problem settings, the discussion of initialization strategies in the learning process is often missed. In this work, we address this important issue of initialization for network embedding that could dramatically improve the performance of the algorithms on both effectiveness and efficiency. Specifically, we first exploit the graph partition technique that divides the graph into several disjoint subsets, and then construct an abstract graph based on the partitions. We obtain the initialization of the embedding for each node in the graph by computing the network embedding on the abstract graph, which is much smaller than the input graph, and then propagating the embedding among the nodes in the input graph. With extensive experiments on various datasets, we demonstrate that our initialization technique significantly improves the performance of the state-of-the-art algorithms on the evaluations of link prediction and node classification by up to 7.76% and 8.74% respectively. Besides, we show that the technique of initialization reduces the running time of the state-of-the-arts by at least 20%.

Motivation & Objective

• Address the critical but often overlooked impact of initialization on network embedding performance.
• Overcome limitations of prior initialization methods that rely on local coarsening and fail to preserve global graph structure.
• Reduce the computational cost of hyperparameter tuning in network embedding algorithms through a learned regression model.
• Improve both the effectiveness (accuracy) and efficiency (training speed) of state-of-the-art network embedding algorithms.

Proposed method

• Apply a graph partitioning algorithm (e.g., METIS) to divide the input graph into disjoint, low-edge-cut subgraphs.
• Construct an abstract graph where each partition becomes a single node and inter-partition edges are weighted by their multiplicity.
• Compute network embeddings on the abstract graph using a modified version of the DeepWalk algorithm with weighted random walks.
• Train a regression model in a preprocessing step to predict optimal hyperparameters (e.g., walk length, negative sample size) for the embedding algorithm.
• Propagate the abstract graph's node embeddings back to the original graph’s nodes via a linear-time message-passing mechanism.
• Use the propagated embeddings as the initial node representations for downstream network embedding algorithms.

Experimental results

Research questions

• RQ1 Can a graph partition-based approach improve the quality of initial node embeddings in network representation learning?
• RQ2 How does the proposed initialization method compare to existing baselines (e.g., HARP, random initialization) in terms of downstream task performance?
• RQ3 To what extent does the proposed method reduce training time for network embedding algorithms?
• RQ4 Can a learned regression model effectively predict optimal hyperparameters for network embedding on abstract graphs without exhaustive search?

Key findings

• GPA improves link prediction performance by up to 7.76% compared to state-of-the-art baselines.
• On node classification, GPA achieves a maximum performance gain of 8.74% over HARP in micro-F1 score.
• The method reduces training time by at least 20% across all evaluated datasets, with a 33.33% reduction observed on the Enron dataset.
• The regression model for hyperparameter tuning enables fast, linear-time configuration without sacrificing embedding quality.
• The abstract graph construction preserves essential structural information, outperforming local coarsening methods that distort global connectivity.
• The initialization strategy leads to faster convergence and better final embeddings, even when used with existing algorithms like LINE and DeepWalk.