[Paper Review] MCMC Louvain for Online Community Detection

Yves Darmaillac, Sébastien Loustau|arXiv (Cornell University)|Dec 5, 2016

Complex Network Analysis Techniques2 references3 citations

TL;DR

This paper proposes MCMC Louvain, a novel online community detection algorithm that dynamically maintains modularity-optimized community structures in evolving networks using a Metropolis-Hastings Markov Chain Monte Carlo framework. By integrating hierarchical aggregation into the proposal mechanism and leveraging conditional probabilities and modularity gain calculations, it enables efficient, real-time clustering updates with ergodic convergence properties, outperforming static baselines in dynamic settings.

ABSTRACT

We introduce a novel algorithm of community detection that maintains dynamically a community structure of a large network that evolves with time. The algorithm maximizes the modularity index thanks to the construction of a randomized hierarchical clustering based on a Monte Carlo Markov Chain (MCMC) method. Interestingly, it could be seen as a dynamization of Louvain algorithm (see Blondel et Al, 2008) where the aggregation step is replaced by the hierarchical instrumental probability.

Motivation & Objective

• To address the challenge of maintaining high-quality community structures in large, dynamically evolving networks.
• To develop a scalable, online algorithm that efficiently updates community partitions as new graph data arrives.
• To overcome limitations of static community detection methods like Louvain by enabling continuous adaptation without full recomputation.
• To integrate hierarchical aggregation into a Metropolis-Hastings framework to improve convergence and sampling efficiency.
• To provide a theoretically grounded, ergodic Markov chain for online modularity maximization with practical computational performance.

Proposed method

• The algorithm uses a Metropolis-Hastings MCMC framework to sample from a target distribution proportional to exp(λQ), where Q is the modularity and λ is a temperature parameter.
• It introduces a hierarchical instrumental prior that enables joint proposal of community reassignments and cluster aggregations across multiple levels of the graph hierarchy.
• Conditional probabilities are explicitly calculated to ensure detailed balance and preserve the stationary distribution, enabling valid MCMC sampling.
• Modularity gain is computed efficiently for local moves (e.g., node reassignment or cluster merging), allowing fast acceptance/rejection decisions.
• The algorithm maintains a sequence of coarsened graphs (G_l) at each level, updating them dynamically when community structures change.
• After each accepted move, higher-level graphs are updated via map-based re-mapping: nodes in the same community are collapsed into single nodes, and edge weights are aggregated.

Experimental results

Research questions

• RQ1 Can a Metropolis-Hastings MCMC framework be effectively adapted to online community detection in dynamic networks?
• RQ2 How can hierarchical aggregation be embedded into the proposal mechanism of an MCMC algorithm to improve convergence and scalability?
• RQ3 What is the impact of using conditional probabilities and modularity gain calculations on the ergodicity and mixing time of the Markov chain?
• RQ4 Can the algorithm maintain high modularity while dynamically adapting to incremental graph changes without full recomputation?
• RQ5 How does the proposed method compare to static community detection algorithms like Louvain in terms of accuracy and computational efficiency on evolving graphs?

Key findings

• The MCMC Louvain algorithm achieves modularity optimization comparable to the original Louvain algorithm in batch settings, validating its core design.
• By integrating aggregation into the proposal mechanism, the algorithm reduces the number of required MCMC iterations, improving convergence speed.
• The use of conditional probabilities ensures detailed balance, guaranteeing that the stationary distribution corresponds to the desired modularity-based target distribution.
• The dynamic update mechanism allows real-time adaptation to graph changes, maintaining consistent community structure across time steps.
• The algorithm supports incremental graph processing, where only affected parts of the hierarchy are updated after each change, reducing computational overhead.
• Preliminary experiments on simulated (preferential attachment) and real-world networks (Koblenz Network Collection) show promising scalability and stability in online settings.