[Paper Review] Comparison and Benchmark of Graph Clustering Algorithms

Lizhen Shi, Bo Chen|arXiv (Cornell University)|May 10, 2020

Complex Network Analysis Techniques71 references4 citations

TL;DR

This paper presents a comprehensive benchmark of over 70 graph clustering algorithms on weighted and unweighted graphs, evaluating both runtime and clustering quality using standardized metrics like NMI and modularity. It identifies Louvain and Infomap as top performers in accuracy, while MCL and LabelRank show high runtime efficiency, offering a practical guide for algorithm selection in network analysis.

ABSTRACT

Graph clustering is widely used in analysis of biological networks, social networks and etc. For over a decade many graph clustering algorithms have been published, however a comprehensive and consistent performance comparison is not available. In this paper we benchmarked more than 70 graph clustering programs to evaluate their runtime and quality performance for both weighted and unweighted graphs. We also analyzed the characteristics of ground truth that affects the performance. Our work is capable to not only supply a start point for engineers to select clustering algorithms but also could provide a viewpoint for researchers to design new algorithms.

Motivation & Objective

• To provide a comprehensive, consistent performance comparison of over 70 graph clustering algorithms, addressing the lack of large-scale, standardized benchmarking in the literature.
• To evaluate both runtime efficiency and clustering quality across diverse graph types, including weighted and unweighted networks.
• To analyze how ground truth characteristics (e.g., community size, mixing parameter) affect algorithm performance.
• To offer a practical reference for engineers and researchers to select or design clustering algorithms based on empirical performance.
• To establish a reproducible benchmark framework using LFR benchmarks and standardized evaluation metrics like NMI and modularity.

Proposed method

• Benchmarked 70+ graph clustering algorithms from major libraries (iGraph, NetworkX, PyCABEM, SNAP, etc.) on synthetic LFR benchmark graphs with varying parameters.
• Used standardized evaluation metrics including Normalized Mutual Information (NMI), modularity, and runtime to assess clustering quality and efficiency.
• Evaluated algorithms across multiple LFR benchmark instances with varying mixing parameters (μ), community sizes, and graph densities.
• Applied statistical ranking (e.g., mean rank across μ values) to compare algorithm performance consistently across all test conditions.
• Reported results in detailed tables showing NMI ranks and runtime (in seconds) for each algorithm across different μ settings.
• Focused on non-overlapping, undirected, unweighted and weighted graphs, with consistent preprocessing and evaluation protocols.

Experimental results

Research questions

• RQ1 Which graph clustering algorithms achieve the highest clustering quality (measured by NMI) across diverse LFR benchmark graphs with varying mixing parameters?
• RQ2 How do the runtime performances of graph clustering algorithms compare across different graph sizes and densities?
• RQ3 How do ground truth characteristics—such as community size distribution and mixing parameter (μ)—influence the performance of clustering algorithms?
• RQ4 Which algorithms consistently rank among the top in both accuracy and efficiency across multiple benchmark configurations?
• RQ5 What are the trade-offs between accuracy (NMI) and computational efficiency (runtime) across the tested algorithms?

Key findings

• Louvain and Infomap consistently achieved the highest NMI scores across all μ values in the LFR benchmark, with mean NMI ranks of 2 and 4, respectively.
• MCL and LabelRank showed the fastest runtime performance, with MCL achieving the lowest mean runtime (10 seconds) on larger graphs, while LabelRank was among the fastest on medium-sized graphs.
• Algorithms like SpectralClustering and AffinityPropagation had high NMI ranks (11–18) and long runtimes (over 20 seconds), indicating poor efficiency-accuracy trade-offs.
• The performance of algorithms like OSLOM and InfohierMap varied significantly with μ, with lower NMI scores at higher mixing parameters (μ > 0.5), indicating sensitivity to community overlap.
• Algorithms such as TopGC and Scan-based methods (e.g., AnyScan) showed high NMI ranks (16–28) and long runtimes (>30 seconds), indicating poor scalability.
• The study revealed that no single algorithm outperforms all others across all metrics: Louvain excelled in accuracy, while MCL and LabelRank led in speed, highlighting the need for algorithm selection based on use-case priorities.