[Paper Review] Degree-degree dependencies in random graphs with heavy-tailed degrees

Remco van der Hofstad, Nelly Litvak|arXiv (Cornell University)|Jan 1, 2014

Complex Network Analysis Techniques42 references21 citations

TL;DR

This paper demonstrates that the Pearson correlation coefficient for degree-degree dependencies in scale-free networks with heavy-tailed degrees is unreliable in large networks, often converging to zero or a non-negative value even under strong negative dependencies. It proposes Spearman’s rank correlation (rho) as a robust alternative, proving it converges to a meaningful limit under general network models, thus enabling consistent analysis of mixing patterns across network sizes.

ABSTRACT

Mixing patterns in large self-organizing networks, such as the Internet, the World Wide Web, social and biological networks are often characterized by degree-degree dependencies between neighbouring nodes. In assortative networks, the degree-degree dependencies are positive (nodes with similar degrees tend to connect to each other), while in disassortative networks, these dependencies are negative. One of the problems with the commonly used Pearson correlation coecient, also known as the assortativity coecient is that its magnitude decreases with the network size in disassortative networks. This makes it impossible to compare mixing patterns, for example, in two web crawls of dierent sizes. As an alternative, we have recently suggested to use rank correlation measures, such as Spearman’s rho. Numerical experiments have conrmed that Spearman’s rho produces consistent values in graphs of dierent sizes but similar structure, and it is able to reveal strong (positive or negative) dependencies in large graphs. In this paper we analytically investigate degree-degree dependencies for scale-free graph sequences. In order to demonstrate the ill behaviour of the Pearson’s correlation coecient, we rst study a simple model of two heavy-tailed highly correlated random variables X and Y , and show that the sample correlation coecient converges in distribution either to a proper random variable on [ 1; 1], or to zero, and the limit is non-negative a.s. if X;Y 0. We next adapt these results to the degree-degree dependencies in networks as described by the Pearson correlation coecient, and show that it is non-negative in the large graph limit when the asymptotic degree distribution has an innite third moment. Furthermore, we provide examples where the Pearson’s correlation coecient converges to zero in a network with strong negative degree-degree dependencies, and another example where this coecient converges in distribution to a random variable. We suggest the alternative degree-degree dependency measure, based on Spearman’s rho, and prove that this statistical estimator converges to an appropriate limit under quite general conditions. These conditions are proved to hold in common network models, such as the conguration model and the preferential attachment model. We conclude that rank correlations provide a suitable and informative method for uncovering network mixing patterns.

Motivation & Objective

• To identify the limitations of the Pearson correlation coefficient in measuring degree-degree dependencies in large, heavy-tailed networks.
• To demonstrate that Pearson’s coefficient can converge to zero or a non-negative value even in networks with strong negative degree correlations.
• To propose Spearman’s rho as a more reliable alternative for measuring degree-degree dependencies in scale-free networks.
• To prove that Spearman’s rho converges to a stable limit under general network models, such as the configuration and preferential attachment models.
• To enable consistent comparison of mixing patterns across networks of different sizes by using rank-based correlation measures.

Proposed method

• Analyze the asymptotic behavior of the sample Pearson correlation coefficient for two heavy-tailed, highly correlated random variables X and Y.
• Adapt the theoretical results on bivariate heavy-tailed distributions to degree-degree dependencies in random graphs.
• Show that the Pearson coefficient converges in distribution to a non-negative random variable or zero when the degree distribution has an infinite third moment.
• Introduce Spearman’s rho as a rank-based alternative to Pearson’s correlation for measuring degree-degree dependencies.
• Prove that Spearman’s rho converges to a well-defined limit under general conditions, including in the configuration model and preferential attachment model.
• Use theoretical convergence results to establish the consistency and reliability of rank correlation for large-scale network analysis.

Experimental results

Research questions

• RQ1 Why does the Pearson correlation coefficient fail to detect strong negative degree-degree dependencies in large scale-free networks?
• RQ2 Under what conditions does the Pearson correlation coefficient converge to zero or a non-negative value in networks with infinite third moments?
• RQ3 Can Spearman’s rho consistently estimate degree-degree dependencies across networks of varying sizes?
• RQ4 What theoretical conditions ensure the convergence of Spearman’s rho to a meaningful limit in random graph models?
• RQ5 How do the convergence properties of Pearson and Spearman correlations differ in networks with heavy-tailed degree distributions?

Key findings

• The Pearson correlation coefficient for degree-degree dependencies converges in distribution to a non-negative random variable or to zero in large networks with infinite third moments, even when strong negative dependencies exist.
• In networks with heavy-tailed degrees and strong negative mixing, the Pearson coefficient can converge to zero, rendering it ineffective for detecting such dependencies.
• Spearman’s rho is shown to converge to a stable limit under general conditions, including in the configuration model and preferential attachment model.
• The convergence of Spearman’s rho is robust across different network sizes, making it suitable for consistent comparison of mixing patterns.
• Theoretical analysis confirms that rank correlation measures are more informative and reliable than Pearson correlation in large-scale, heavy-tailed networks.
• The study establishes that Spearman’s rho provides a consistent and interpretable measure of network mixing patterns, even when Pearson’s coefficient fails.