[Paper Review] On the Stability of Community Detection Algorithms on Longitudinal Citation Data

Michael James Bommarito, Daniel Katz+1 more|ArXiv.org|Aug 4, 2009

Complex Network Analysis Techniques29 references4 citations

TL;DR

This paper evaluates the stability of three community detection algorithms—eigenvector, fast-greedy, and edge-betweenness—on simulated longitudinal citation networks, which are directed and acyclic. It finds that algorithm performance varies significantly with network density and preferential attachment, with betweenness showing high stability under high attachment, while modularity-based methods (eigenvector and fast-greedy) form a consistent stability curve, suggesting context-dependent algorithm selection is essential for reliable community detection in citation data.

ABSTRACT

There are fundamental differences between citation networks and other classes of graphs. In particular, given that citation networks are directed and acyclic, methods developed primarily for use with undirected social network data may face obstacles. This is particularly true for the dynamic development of community structure in citation networks. Namely, it is neither clear when it is appropriate to employ existing community detection approaches nor is it clear how to choose among existing approaches. Using simulated data, we attempt to clarify the conditions under which one should use existing methods and which of these algorithms is appropriate in a given context. We hope this paper will serve as both a useful guidepost and an encouragement to those interested in the development of more targeted approaches for use with longitudinal citation data.

Motivation & Objective

• To assess the stability and reliability of community detection algorithms on longitudinal citation networks, which are inherently directed and acyclic.
• To identify conditions under which existing community detection methods are appropriate for citation data, given structural differences from undirected social networks.
• To guide researchers in selecting among eigenvector, fast-greedy, and edge-betweenness algorithms based on network characteristics such as density and preferential attachment.
• To provide diagnostic heuristics for applied researchers using community detection in citation networks, emphasizing algorithmic stability and accuracy.
• To encourage the development of more targeted community detection methods tailored to the unique properties of citation networks.

Proposed method

• The study uses a stochastic model to generate synthetic directed acyclic citation networks with tunable parameters, including growth rate (λ), attachment rate (μ), and preferential attachment strength (α).
• The model simulates longitudinal citation processes, allowing for controlled variation in network density and attachment dynamics to test algorithm behavior.
• Three canonical community detection algorithms—eigenvector, fast-greedy, and edge-betweenness—are applied to each simulated network to assess stability and community count.
• Stability is measured as the mean and standard deviation of adjusted Rand index (ARI) across multiple runs, with higher stability indicating consistent community structure.
• The analysis compares algorithm performance across varying network conditions, particularly focusing on the impact of preferential attachment (α = 0 vs. α = 1) and network density (λ/μ ratio).
• Results are visualized via scatter plots of stability versus number of detected communities, revealing functional relationships between algorithm behavior and network structure.

Experimental results

Research questions

• RQ1 Under what conditions is it appropriate to apply existing community detection algorithms to longitudinal citation networks?
• RQ2 How does preferential attachment affect the stability and consistency of community detection algorithms in citation networks?
• RQ3 How do the stability and number of communities detected vary across eigenvector, fast-greedy, and edge-betweenness algorithms under different network densities?
• RQ4 Is there a functional relationship between algorithm stability and the number of detected communities across different network configurations?
• RQ5 Can diagnostic heuristics be derived to guide researchers in selecting the most stable and appropriate algorithm for a given citation network?

Key findings

• Edge-betweenness showed high stability (mean stability 0.95, std 0.020) and low variance when preferential attachment was strong (α = 1), particularly in less dense, rapidly growing networks.
• Fast-greedy and eigenvector algorithms maintained a consistent stability curve across varying network conditions, with fast-greedy showing higher mean stability (0.80) than eigenvector (0.70).
• In high-density, slowly growing networks (λ=1, μ=4), betweenness had lower stability (0.80) and higher variance (0.43) compared to fast-greedy and eigenvector.
• When preferential attachment was introduced (α=1), betweenness collapsed onto the stability curve of eigenvector and fast-greedy, indicating convergence under strong attachment.
• In rapidly growing, less dense networks (λ=4, μ=1), betweenness achieved higher stability (0.91) with more communities (39.85 mean), suggesting density and growth rate influence algorithm performance.
• The slope of the stability vs. community count relationship appears to depend on the μ/λ ratio, indicating a potential functional form governing algorithm behavior across network types.