[Paper Review] Characterizing Directed and Undirected Networks via Multidimensional Walks with Jumps

Fabrício Murai, Bruno Ribeiro+2 more|arXiv (Cornell University)|Mar 23, 2017

Complex Network Analysis Techniques21 references4 citations

TL;DR

This paper proposes Directed Unbiased Frontier Sampling (DUFS), a novel random walk-based method for sampling both directed and undirected networks by using multiple coordinated walkers with random jumps to prevent trapping. DUFS constructs an undirected graph in real time from walker trajectories and provides an improved, minimum variance unbiased estimator for node label distributions, significantly outperforming existing methods in estimating the head of the out-degree distribution and matching or exceeding accuracy in the tail.

ABSTRACT

Estimating distributions of node characteristics (labels) such as number of connections or citizenship of users in a social network via edge and node sampling is a vital part of the study of complex networks. Due to its low cost, sampling via a random walk (RW) has been proposed as an attractive solution to this task. Most RW methods assume either that the network is undirected or that walkers can traverse edges regardless of their direction. Some RW methods have been designed for directed networks where edges coming into a node are not directly observable. In this work, we propose Directed Unbiased Frontier Sampling (DUFS), a sampling method based on a large number of coordinated walkers, each starting from a node chosen uniformly at random. It is applicable to directed networks with invisible incoming edges because it constructs, in real-time, an undirected graph consistent with the walkers trajectories, and due to the use of random jumps which prevent walkers from being trapped. DUFS generalizes previous RW methods and is suited for undirected networks and to directed networks regardless of in-edges visibility. We also propose an improved estimator of node label distributions that combines information from the initial walker locations with subsequent RW observations. We evaluate DUFS, compare it to other RW methods, investigate the impact of its parameters on estimation accuracy and provide practical guidelines for choosing them. In estimating out-degree distributions, DUFS yields significantly better estimates of the head of the distribution than other methods, while matching or exceeding estimation accuracy of the tail. Last, we show that DUFS outperforms uniform node sampling when estimating distributions of node labels of the top 10% largest degree nodes, even when sampling a node uniformly has the same cost as RW steps.

Motivation & Objective

• To address the challenge of sampling directed networks where incoming edges are invisible, preventing standard random walks from exploring the full graph.
• To generalize existing random walk sampling methods to work for both directed and undirected networks, including cases with hidden in-edges.
• To develop an improved estimator for node label distributions that combines initial walker locations with subsequent walk observations.
• To provide practical guidelines for choosing DUFS parameters to optimize estimation accuracy.
• To demonstrate that DUFS outperforms uniform node sampling even when the cost of a single uniform node query equals that of a random walk step.

Proposed method

• DUFS employs multiple coordinated random walkers, each starting from a node chosen uniformly at random.
• Each walker performs a random walk with random jumps to avoid getting trapped in strongly connected components.
• The method dynamically constructs an undirected graph from the trajectories of all walkers, enabling unbiased sampling even in directed networks with invisible incoming edges.
• It introduces a new estimator that combines information from initial walker positions and subsequent walk observations, using the sufficient and complete statistic $ n_i + m_i $.
• The estimator is proven to be the minimum variance unbiased estimator (MVUE) via the Lehmann-Scheffé theorem, using the Fisher-Neyman factorization and completeness of $ n_i + m_i $.
• The method generalizes Frontier Sampling (FS) and Directed Unbiased Random Walk (DURW), unifying their strengths for both undirected and directed networks.

Experimental results

Research questions

• RQ1 Can a random walk-based sampling method be designed to work effectively in directed networks where incoming edges are not observable?
• RQ2 How can random walks be modified to avoid trapping in strongly connected components while maintaining unbiased estimation of node label distributions?
• RQ3 What estimator structure yields the minimum variance unbiased estimate of node label distributions in such sampling frameworks?
• RQ4 How do the parameters of the sampling process (e.g., number of walkers, jump probability) affect estimation accuracy?
• RQ5 Does DUFS outperform uniform node sampling in terms of estimation accuracy for high-degree nodes, even when sampling costs are equivalent?

Key findings

• DUFS yields significantly better estimates of the head of the out-degree distribution than other random walk methods.
• DUFS matches or exceeds the estimation accuracy of the tail of the out-degree distribution compared to existing methods.
• The proposed estimator is proven to be the minimum variance unbiased estimator (MVUE), with theoretical guarantees based on sufficient and complete statistics.
• DUFS outperforms uniform node sampling in estimating distributions of labels for the top 10% of largest degree nodes, even when the cost of a single uniform node query equals that of a random walk step.
• The method is robust across both directed and undirected networks and generalizes prior approaches like Frontier Sampling and DURW.
• Parameter sensitivity analysis provides practical guidelines for tuning the number of walkers and jump probability to optimize estimation accuracy.