[Paper Review] Bayesian Overlapping Community Detection in Dynamic Networks

Mahsa Ghorbani, Hamid R. Rabiee+1 more|arXiv (Cornell University)|May 8, 2016

Complex Network Analysis Techniques36 references3 citations

TL;DR

This paper proposes DBOCD, a Bayesian non-parametric method for detecting overlapping communities in dynamic networks without prior knowledge of community count. It uses link communities and the Recurrent Chinese Restaurant Process (RCRP) to infer soft node memberships and maintain temporal consistency, outperforming state-of-the-art methods on synthetic and real-world datasets in terms of modularity and stability.

ABSTRACT

Detecting community structures in social networks has gained considerable attention in recent years. However, lack of prior knowledge about the number of communities, and their overlapping nature have made community detection a challenging problem. Moreover, many of the existing methods only consider static networks, while most of real world networks are dynamic and evolve over time. Hence, finding consistent overlapping communities in dynamic networks without any prior knowledge about the number of communities is still an interesting open research problem. In this paper, we present an overlapping community detection method for dynamic networks called Dynamic Bayesian Overlapping Community Detector (DBOCD). DBOCD assumes that in every snapshot of network, overlapping parts of communities are dense areas and utilizes link communities instead of common node communities. Using Recurrent Chinese Restaurant Process and community structure of the network in the last snapshot, DBOCD simultaneously extracts the number of communities and soft community memberships of nodes while maintaining the consistency of communities over time. We evaluated DBOCD on both synthetic and real dynamic data-sets to assess its ability to find overlapping communities in different types of network evolution. The results show that DBOCD outperforms the recent state of the art dynamic community detection methods.

Motivation & Objective

• To address the challenge of detecting overlapping communities in dynamic networks without prior knowledge of the number of communities.
• To maintain temporal consistency of communities across network snapshots while handling dynamic changes like node/edge additions and deletions.
• To reduce computational complexity by using link partitioning instead of node partitioning.
• To develop a scalable, non-parametric Bayesian model that infers soft memberships and community counts simultaneously.
• To evaluate the method on diverse network evolution patterns, including merging, splitting, and birth/death of communities.

Proposed method

• Uses a generative model based on link communities rather than node communities to improve scalability and handle overlapping memberships.
• Employs the Recurrent Chinese Restaurant Process (RCRP) as a prior to model community evolution and infer the number of communities dynamically.
• Infers community memberships and community counts via Bayesian inference on adjacency matrices from discrete network snapshots.
• Leverages the community structure from the previous time step as a prior to ensure temporal consistency.
• Applies statistical inference methods to extract soft memberships and dynamically adjust community counts.
• Uses a non-parametric approach to avoid requiring the number of communities as input.

Experimental results

Research questions

• RQ1 Can a Bayesian non-parametric model detect overlapping communities in dynamic networks without prior knowledge of the number of communities?
• RQ2 How well does the method maintain community consistency across time steps in the presence of network evolution?
• RQ3 Does link-based community detection outperform node-based approaches in terms of scalability and accuracy for overlapping communities?
• RQ4 How does the method perform on diverse network evolution patterns such as merging, splitting, and community birth/death?
• RQ5 Can the method achieve stable and high modularity on real-world dynamic networks without ground truth?

Key findings

• DBOCD outperforms state-of-the-art methods like AFFECT, FacetNet, and DSBM in terms of modularity and stability on synthetic datasets with various evolution patterns.
• On the birth, death, and birth-death processes, DBOCD accurately captures changes in the number of communities, while AFFECT fails to track the gradient of such changes.
• In the split and merge processes, DBOCD shows stable performance despite abrupt changes, though it slightly underestimates the number of communities due to abrupt transitions.
• On the NEC blog dataset, DBOCD maintains stable modularity over time, while other methods show significant fluctuations.
• On the DBLP co-authorship dataset, DBOCD achieves higher modularity than FacetNet and DSBM, even though DSBM is non-overlapping, indicating DBOCD's robustness on both overlapping and non-overlapping networks.
• The method demonstrates consistent performance across both increasing (DBLP) and decreasing (NEC blog) network sizes, confirming its adaptability to diverse real-world dynamics.