[论文解读] Fast and scalable likelihood maximization for Exponential Random Graph Models

Nicolò Vallarano, Matteo Bruno+6 more|arXiv (Cornell University)|Jan 29, 2021

Complex Network Analysis Techniques被引用 5

一句话总结

本文提出并基准测试了三种算法——牛顿法、拟牛顿法和定点迭代法——用于指数随机图模型（ERGMs）的似然最大化，解决了精度、速度和可扩展性问题。定点迭代法使包含数十万个节点的大规模网络在数秒内实现收敛，其可扩展性优于其他方法，同时保持了高精度。

ABSTRACT

Exponential Random Graph Models (ERGMs) have gained increasing popularity over the years. Rooted into statistical physics, the ERGMs framework has been successfully employed for reconstructing networks, detecting statistically significant patterns in graphs, counting networked configurations with given properties. From a technical point of view, the ERGMs workflow is defined by two subsequent optimization steps: the first one concerns the maximization of Shannon entropy and leads to identify the functional form of the ensemble probability distribution that is maximally non-committal with respect to the missing information; the second one concerns the maximization of the likelihood function induced by this probability distribution and leads to its numerical determination. This second step translates into the resolution of a system of $O(N)$ non-linear, coupled equations (with $N$ being the total number of nodes of the network under analysis), a problem that is affected by three main issues, i.e. accuracy, speed and scalability. The present paper aims at addressing these problems by comparing the performance of three algorithms (i.e. Newton's method, a quasi-Newton method and a recently-proposed fixed-point recipe) in solving several ERGMs, defined by binary and weighted constraints in both a directed and an undirected fashion. While Newton's method performs best for relatively little networks, the fixed-point recipe is to be preferred when large configurations are considered, as it ensures convergence to the solution within seconds for networks with hundreds of thousands of nodes (e.g. the Internet, Bitcoin). We attach to the paper a Python code implementing the three aforementioned algorithms on all the ERGMs considered in the present work.

研究动机与目标

• 解决指数随机图模型（ERGMs）中似然最大化所面临的计算挑战，即求解O(N)个非线性耦合方程。
• 提升大规模网络（如互联网或比特币网络）中ERGM推断的可扩展性，使传统方法变得不可行。
• 评估并比较三种数值优化算法——牛顿法、拟牛顿法和定点迭代法——在二值与加权、有向与无向ERGM中的性能。
• 提供一个实用的开源Python实现，以支持网络科学研究中的可复现性与应用。

提出的方法

• 实现牛顿法用于ERGM中的似然最大化，利用二阶导数在小到中等规模网络上实现快速局部收敛。
• 应用拟牛顿法近似Hessian矩阵，降低计算成本，同时保持合理的收敛速度。
• 采用一种新提出的定点迭代方法，避免Hessian矩阵的计算，从而在大规模网络上实现稳定收敛。
• 将似然最大化问题表述为求解由对数似然函数梯度导出的O(N)个非线性方程组。
• 在包含二值与加权约束的多种ERGM上测试所有三种算法，涵盖有向与无向网络拓扑。
• 发布所有三种算法的Python实现，覆盖所有考虑的ERGM，以确保可复现性与可用性。

实验结果

研究问题

• RQ1 在ERGM似然最大化中，牛顿法、拟牛顿法与定点迭代法在收敛速度和精度方面如何比较？
• RQ2 哪种算法在包含数十万个节点的大规模网络（如互联网或比特币）中可扩展性最佳？
• RQ3 定点迭代法是否能在牛顿法计算成本过高而不可行的大规模ERGM中实现数秒内可靠收敛？
• RQ4 在二值与加权网络模型中，三种算法在计算成本与收敛质量之间的权衡如何？
• RQ5 各算法在有向与无向网络配置下的性能表现有何差异？

主要发现

• 牛顿法在相对较小的网络中实现了最高精度和最快收敛，但其计算成本随网络规模增长而急剧上升。
• 定点迭代法使包含数十万个节点的网络（如互联网或比特币）在数秒内实现收敛。
• 在可扩展性方面，定点迭代法优于牛顿法与拟牛顿法，能在大规模ERGM上保持稳定收敛。
• 对于大规模网络，定点迭代法为牛顿法提供了实用替代方案，避免了昂贵的Hessian矩阵计算，同时确保数值稳定性。
• 所提出的Python实现支持所有测试的ERGM，包括二值与加权、有向与无向模型，实现了广泛的可复现性与应用性。