[Paper Review] A Comprehensive Review of Community Detection in Graphs

J Li, Lai, Songning+12 more|arXiv (Cornell University)|Sep 21, 2023

Complex Network Analysis Techniques4 citations

TL;DR

This paper presents a comprehensive review of community detection in graphs, introducing the Revised Medoid-Shift (RMS) algorithm that combines K-nearest neighbors (KNN) and medoid-shift principles to improve community detection performance on non-Euclidean graph structures. The RMS method achieves enhanced modularity and Normalized Mutual Information (NMI) scores, offering a robust, machine learning-compatible solution for complex network analysis.

ABSTRACT

The study of complex networks has significantly advanced our understanding of community structures which serves as a crucial feature of real-world graphs. Detecting communities in graphs is a challenging problem with applications in sociology, biology, and computer science. Despite the efforts of an interdisciplinary community of scientists, a satisfactory solution to this problem has not yet been achieved. This review article delves into the topic of community detection in graphs, which serves as a thorough exposition of various community detection methods from perspectives of modularity-based method, spectral clustering, probabilistic modelling, and deep learning. Along with the methods, a new community detection method designed by us is also presented. Additionally, the performance of these methods on the datasets with and without ground truth is compared. In conclusion, this comprehensive review provides a deep understanding of community detection in graphs.

Motivation & Objective

• To provide a comprehensive review of community detection methods across diverse domains such as sociology, biology, and computer science.
• To address the persistent challenge of detecting meaningful community structures in complex, real-world graphs with strong internal and weak inter-cluster connections.
• To introduce and validate a novel community detection algorithm, Revised Medoid-Shift (RMS), designed to overcome limitations of traditional machine learning methods on graph-structured data.
• To evaluate the performance of RMS using standard metrics like modularity and NMI, demonstrating its superiority over existing approaches.
• To bridge interdisciplinary research by integrating methods from statistical physics and data mining into a unified framework for community detection.

Proposed method

• The Revised Medoid-Shift (RMS) algorithm integrates the medoid-shift clustering principle with K-nearest neighbors (KNN) to identify dense regions in graph space.
• RMS operates directly on graph structures by leveraging node proximity and local density, avoiding reliance on Euclidean distance assumptions.
• The method uses a kernel-based density estimation approach where medoids serve as cluster centers, updated iteratively based on the highest-density neighbors within KNN neighborhoods.
• It incorporates a revised shift vector that accounts for graph topology, enabling effective clustering in non-Euclidean, network-based data.
• The algorithm is evaluated using standard community detection metrics: modularity (Q) and Normalized Mutual Information (NMI), with results compared against baseline methods.
• The RMS framework is designed to be scalable and adaptable to various network types, including social, biological, and information networks.

Experimental results

Research questions

• RQ1 How can community detection be effectively applied to non-Euclidean graph structures where traditional machine learning methods fail?
• RQ2 What improvements can be achieved by integrating KNN with medoid-shift principles in community detection algorithms?
• RQ3 To what extent does the Revised Medoid-Shift (RMS) method outperform existing community detection algorithms in terms of modularity and NMI?
• RQ4 How does the RMS algorithm handle complex, real-world network topologies such as those found in social and biological systems?
• RQ5 What are the key limitations of current community detection methods, and how does RMS address them?

Key findings

• The Revised Medoid-Shift (RMS) algorithm demonstrates significant improvements in modularity and Normalized Mutual Information (NMI) compared to baseline methods on benchmark graph datasets.
• RMS effectively detects communities in non-Euclidean graph structures, overcoming the inherent limitations of mean-shift and other Euclidean-based clustering algorithms.
• The integration of KNN with medoid-shift enables the algorithm to preserve local graph topology and identify dense, well-separated communities.
• Empirical evaluation confirms that RMS achieves higher clustering accuracy and stability across diverse network types, including social and biological networks.
• The method shows robustness to noise and varying network densities, making it suitable for real-world applications.
• The review identifies four main frameworks of community detection and highlights the strengths and weaknesses of each, providing a foundation for future algorithmic development.