[Paper Review] Network Vector: Distributed Representations of Networks with Global Context

Hao Wu, Kristina Lerman|arXiv (Cornell University)|Sep 7, 2017

Advanced Graph Neural Networks19 references3 citations

TL;DR

Network Vector is a neural embedding algorithm that learns joint distributed representations of nodes and entire networks by capturing both local neighborhood structure and global network context through random walk-based training. It outperforms state-of-the-art methods like node2vec in node classification, role discovery, and concept analogy tasks by explicitly modeling the network's global structure, achieving up to 10.3% higher Micro-F1 score with limited labeled data.

ABSTRACT

We propose a neural embedding algorithm called Network Vector, which learns distributed representations of nodes and the entire networks simultaneously. By embedding networks in a low-dimensional space, the algorithm allows us to compare networks in terms of structural similarity and to solve outstanding predictive problems. Unlike alternative approaches that focus on node level features, we learn a continuous global vector that captures each node's global context by maximizing the predictive likelihood of random walk paths in the network. Our algorithm is scalable to real world graphs with many nodes. We evaluate our algorithm on datasets from diverse domains, and compare it with state-of-the-art techniques in node classification, role discovery and concept analogy tasks. The empirical results show the effectiveness and the efficiency of our algorithm.

Motivation & Objective

• To address the limitation of existing network embedding methods that focus only on local node neighborhoods and fail to capture global network structure.
• To develop a scalable, end-to-end algorithm that jointly learns node and network-level representations in a low-dimensional space.
• To enable quantitative comparison of network structural similarity and identification of structurally analogous nodes or networks.
• To improve performance on predictive tasks such as node classification, role discovery, and concept analogy by incorporating global context.
• To provide a flexible framework that can be extended to include textual or non-textual attributes in network representation learning.

Proposed method

• The algorithm uses random walks to generate sequences of nodes, treating each walk as a 'sentence' and the network as a 'document'.
• It introduces a global context vector that is jointly optimized with node embeddings to maximize the likelihood of predicting the next node in a random walk sequence.
• The objective function combines local node proximity (via skip-gram model) with global network context, using negative sampling for efficient training.
• The global context vector is shared across all nodes and captures structural properties of the entire network.
• The method is scalable to large real-world networks and supports both node-level and network-level similarity comparisons.
• It extends the Paragraph Vector framework (Le & Mikolov, 2014) to networks by replacing words with nodes and sentences with random walk paths.

Experimental results

Research questions

• RQ1 Can a unified embedding framework jointly represent nodes and the entire network while preserving both local and global structural information?
• RQ2 How does incorporating global network context improve performance on node classification and role discovery tasks compared to local-only methods?
• RQ3 To what extent does Network Vector outperform state-of-the-art methods like node2vec, LINE, and DeepWalk in capturing structural similarity across diverse network types?
• RQ4 How does the algorithm perform under varying levels of labeled data, particularly in low-data regimes?
• RQ5 In what scenarios does global context provide the most benefit, and when is it less effective?

Key findings

• Network Vector consistently outperforms node2vec across all datasets in both Macro-F1 and Micro-F1 scores, with performance gains increasing under BFS-like random walk sampling (high q values).
• On the BlogCatalog dataset, Network Vector achieves a 10.3% improvement in Micro-F1 score over node2vec when 40% of the data is labeled, indicating strong generalization under limited supervision.
• On the PPI dataset, Network Vector maintains stable performance gains over node2vec across different (p,q) settings, suggesting robustness to sampling bias.
• With only 10% labeled data, Network Vector still achieves a 1.5% gain in Macro-F1 and 7.1% in Micro-F1 over node2vec, demonstrating effectiveness in low-data regimes.
• The performance gain is most pronounced when random walks favor BFS-like exploration, indicating that global context is most beneficial when local information is dense and structured.
• The algorithm effectively enables network-level comparison, allowing identification of structurally similar nodes (e.g., managers in organizational networks) and analogous relationships across different networks.