[Paper Review] A Statistical Density-Based Analysis of Graph Clustering Algorithm Performance

Pierre Miasnikof, Alexander Y. Shestopaloff+3 more|arXiv (Cornell University)|Jun 6, 2019

Complex Network Analysis Techniques53 references4 citations

TL;DR

This paper proposes a statistical, density-based framework for objectively assessing graph clustering quality by comparing global, intra-cluster, and inter-cluster densities. Using a non-generative null model and a modified t-test, the method provides statistically significant, axiomatic, and robust evaluation—outperforming modularity and conductance in sensitivity, numerical stability, and consistency with clustering quality axioms.

ABSTRACT

Measuring graph clustering quality remains an open problem. To address it, we introduce quality measures based on comparisons of intra- and inter-cluster densities, an accompanying statistical test of the significance of their differences and a step-by-step routine for clustering quality assessment. Our null hypothesis does not rely on any generative model for the graph, unlike modularity which uses the configuration model as a null model. Our measures are shown to meet the axioms of a good clustering quality function, unlike the very commonly used modularity measure. They also have an intuitive graph-theoretic interpretation, a formal statistical interpretation and can be easily tested for significance. Our work is centered on the idea that well clustered graphs will display a significantly larger intra-cluster density than inter-cluster density. We develop tests to validate the existence of such a cluster structure. We empirically explore the behavior of our measures under a number of stress test scenarios and compare their behavior to the commonly used modularity and conductance measures. Empirical stress test results confirm that our measures compare very favorably to the established ones. In particular, they are shown to be more responsive to graph structure and less sensitive to sample size and breakdowns during numerical implementation and less sensitive to uncertainty in connectivity. These features are especially important in the context of larger data sets or when the data may contain errors in the connectivity patterns.

Motivation & Objective

• To address the open problem of measuring graph clustering quality without relying on generative null models.
• To define a rigorous, statistically grounded quality function that satisfies axioms of good clustering.
• To develop a significance testing framework that is robust to numerical instability and sparse connectivity.
• To compare the performance of clustering algorithms objectively using density-based metrics and statistical inference.
• To overcome limitations of modularity and conductance, such as insensitivity to structure and failure under stress conditions.

Proposed method

• The method defines three key density measures: global density (K), mean intra-cluster density (K_intra), and mean inter-cluster density (K_inter).
• It introduces a statistical test based on a non-generative null model where cluster assignments are randomized, avoiding reliance on models like the configuration model.
• A modified two-sample t-test is applied to compare K_intra and K_inter, with standard error estimated via Monte Carlo resampling to handle degenerate cases (e.g., zero inter-cluster edges).
• The test evaluates the null hypothesis H₀: K_intra = K_inter, with rejection indicating statistically significant clustering.
• The framework includes a two-algorithm heuristic test using p-value differences to rank clustering quality between competing algorithms.
• The approach is validated through stress tests on synthetic and real graphs, comparing results to modularity and conductance.

Experimental results

Research questions

• RQ1 Can a clustering quality measure be developed that is both statistically rigorous and independent of generative graph models?
• RQ2 How does the proposed method compare to modularity and conductance in sensitivity to graph structure and numerical robustness?
• RQ3 Does the method satisfy the axioms of a good clustering quality function, such as monotonicity and consistency?
• RQ4 Can statistical significance be reliably assessed even when inter-cluster edge counts are zero or near-zero?
• RQ5 Is the method capable of distinguishing meaningful clusterings from random or poor ones in edge-case scenarios?

Key findings

• The proposed method demonstrated superior responsiveness to actual clustering structure compared to modularity and conductance, especially in stress test scenarios.
• For the Louvain algorithm, the p-value was approximately 0.0, leading to strong rejection of the null hypothesis and confirmation of statistically significant clustering.
• For the ALP algorithm, the p-value was approximately 0.1, failing to reject the null, indicating no significant difference between intra- and inter-cluster densities.
• The method successfully computed a valid t-statistic and p-value for ALP despite zero inter-cluster edges, due to its Monte Carlo-based standard error estimation.
• The two-algorithm heuristic test showed a p-value difference of approximately 0.10, providing additional evidence that Louvain produced better clusters than ALP.
• The method was found to be less prone to numerical breakdowns and more consistent with the axioms of good clustering quality functions than modularity.