[Paper Review] A note on quickly sampling a sparse matrix with low rank expectation

Karl Rohe, Jun Tao+2 more|arXiv (Cornell University)|Mar 8, 2017

Complex Network Analysis Techniques10 references3 citations

TL;DR

This paper introduces fastRG, an efficient algorithm for sampling sparse random graphs with low-rank expectation $E(A) = XSY^T$, where $X$, $S$, and $Y$ are non-negative matrices. By first sampling the total number of edges $m$ from a Poisson distribution and then sampling each edge via weighted probabilities derived from normalized matrices $\tilde{X}$, $\tilde{S}$, and $\tilde{Y}$, fastRG achieves $O(n)$ time complexity for sparse graphs ($m = O(n)$), significantly outperforming the naive $O(n^2)$ element-wise approach.

ABSTRACT

Given matrices $X,Y \in R^{n imes K}$ and $S \in R^{K imes K}$ with positive elements, this paper proposes an algorithm fastRG to sample a sparse matrix $A$ with low rank expectation $E(A) = XSY^T$ and independent Poisson elements. This allows for quickly sampling from a broad class of stochastic blockmodel graphs (degree-corrected, mixed membership, overlapping) all of which are specific parameterizations of the generalized random product graph model defined in Section 2.2. The basic idea of fastRG is to first sample the number of edges $m$ and then sample each edge. The key insight is that because of the the low rank expectation, it is easy to sample individual edges. The naive "element-wise" algorithm requires $O(n^2)$ operations to generate the $n imes n$ adjacency matrix $A$. In sparse graphs, where $m = O(n)$, ignoring log terms, fastRG runs in time $O(n)$. An implementation in fastRG is available on github. A computational experiment in Section 2.4 simulates graphs up to $n=10,000,000$ nodes with $m = 100,000,000$ edges. For example, on a graph with $n=500,000$ and $m = 5,000,000$, fastRG runs in less than one second on a 3.5 GHz Intel i5.

Motivation & Objective

• To address the computational bottleneck in simulating random graphs with low-rank expectations, particularly for large-scale network analysis.
• To develop an efficient sampling algorithm that avoids the $O(n^2)$ cost of element-wise generation in sparse graphs.
• To enable fast simulation of diverse stochastic blockmodel classes (e.g., degree-corrected, mixed membership, overlapping SBMs) under a unified generalized random product graph model.
• To achieve near-linear time sampling performance for sparse graphs with $m = O(n)$ edges.

Proposed method

• FastRG samples the total number of edges $m$ from a Poisson distribution with rate $\sum_{u,v} \tilde{S}_{uv}$, where $\tilde{S} = C_X S C_Y$.
• It normalizes the latent feature matrices $X$ and $Y$ into $\tilde{X} = X C_X^{-1}$ and $\tilde{Y} = Y C_Y^{-1}$ to define edge sampling probabilities.
• Each edge is sampled in sequence by first selecting latent blocks $U \in \{1,\dots,K_x\}$ and $V \in \{1,\dots,K_y\}$ with probabilities proportional to $\tilde{S}_{uv}$.
• Then, nodes $I$ and $J$ are selected with probabilities $\tilde{X}_{iU}$ and $\tilde{Y}_{jV}$, respectively, allowing for multiple edges.
• The algorithm supports directed, multi-edge, and self-loop graphs, which can be transformed into simple graphs via post-processing.
• Theoretical justification relies on conditioning on the total edge count and applying a classical result that a vector of independent Poisson variables becomes multinomial when conditioned on their sum.

Experimental results

Research questions

• RQ1 Can we sample sparse random graphs with low-rank expectation in $O(n)$ time instead of $O(n^2)$?
• RQ2 How can we efficiently generate graphs from generalized random product graph models, including degree-corrected and overlapping stochastic blockmodels?
• RQ3 What is the computational performance gain of fastRG over naive element-wise sampling in large-scale networks?
• RQ4 Can fastRG be used to approximate Bernoulli-edge graphs via Poisson edge sampling with controlled error?

Key findings

• FastRG runs in $O(n)$ time for sparse graphs with $m = O(n)$ edges, achieving near-linear performance.
• On a graph with $n = 500,000$ nodes and $m = 5,000,000$ edges, fastRG completes in under one second on a 3.5 GHz Intel i5 processor.
• The algorithm correctly samples from the generalized random product graph model, with $E(A) = XSY^T$, and ensures that edge sampling probabilities match the multinomial distribution conditioned on total edge count.
• Theoretical analysis confirms that fastRG generates graphs with the correct distribution, as shown by proving $\mathbb{P}(A = a) = \mathbb{P}(\tilde{A} = a)$ for any adjacency matrix $a$.
• The method enables accurate approximation of Bernoulli-edge graphs via Poisson edge sampling, with expected $L^2$ error $O(\alpha_n^2)$, where $\alpha_n$ is the average degree.
• An R implementation is publicly available on GitHub, enabling reproducibility and integration into network simulation pipelines.