[Paper Review] Informative core identification in complex networks

Ruizhong Miao, Tianxi Li|arXiv (Cornell University)|Jan 16, 2021

Complex Network Analysis Techniques56 references4 citations

TL;DR

This paper proposes a novel core-periphery model that identifies informative core structures in complex networks without assuming a specific form for the core, using spectral algorithms with strong theoretical guarantees. The method outperforms traditional approaches in core identification, enabling more accurate hierarchical community detection in citation networks by filtering non-informative periphery nodes.

ABSTRACT

In network analysis, the core structure of modeling interest is usually hidden in a larger network in which most structures are not informative. The noise and bias introduced by the non-informative component in networks can obscure the salient structure and limit many network modeling procedures' effectiveness. This paper introduces a novel core-periphery model for the non-informative periphery structure of networks without imposing a specific form for the informative core structure. We propose spectral algorithms for core identification as a data preprocessing step for general downstream network analysis tasks based on the model. The algorithm enjoys a strong theoretical guarantee of accuracy and is scalable for large networks. We evaluate the proposed method by extensive simulation studies demonstrating various advantages over many traditional core-periphery methods. The method is applied to extract the informative core structure from a citation network and give more informative results in the downstream hierarchical community detection.

Motivation & Objective

• To address the challenge of non-informative network structures obscuring salient patterns in large-scale networks.
• To develop a principled preprocessing method that identifies informative cores while filtering out uninformative periphery nodes.
• To propose a core-periphery model that does not constrain the core to a specific network model, enhancing generality for downstream analysis.
• To provide theoretical guarantees for core identification accuracy under mild conditions.
• To evaluate the method on real citation networks and demonstrate improved performance in hierarchical community detection.

Proposed method

• Introduces a core-periphery model where the core contains informative connection patterns and the periphery is non-informative, without assuming a specific model for the core.
• Develops spectral algorithms for core identification based on the model, leveraging eigenstructure of the network adjacency matrix.
• Uses eigengap analysis to detect the rank of the informative core, with the core identified as the leading eigenspace when a significant eigengap exists.
• Applies two variants: an Erdős–Rényi-type model and a configuration model for the periphery, both yielding robust core detection.
• Employs a two-step process: first, identify the core via spectral clustering on the adjacency matrix; second, apply hierarchical community detection (HCD) on the core.
• Uses cosine similarity between paper abstracts to construct a ground-truth similarity matrix for evaluating hierarchical structure quality.

Experimental results

Research questions

• RQ1 Can a core-periphery model be designed that does not assume a specific parametric form for the informative core, enabling broader applicability?
• RQ2 How can spectral methods be used to identify the core with theoretical consistency guarantees in the presence of non-informative periphery structures?
• RQ3 To what extent does core extraction improve the interpretability and accuracy of downstream hierarchical community detection in citation networks?
• RQ4 How does the proposed method compare to traditional core identification techniques like k-core, PageRank, and degree-based pruning in capturing meaningful network hierarchies?
• RQ5 Can the method robustly identify cores even when the periphery is large and noisy, preserving the true underlying structure?

Key findings

• The proposed method achieved a Spearman correlation of 0.350 between network-derived and abstract-based hierarchical structures when $N_{Σ} = 635$, significantly outperforming benchmarks like k-core (0.167) and degree (0.099).
• The configuration-type model yielded a correlation of 0.350, while the ER-type model achieved 0.340, indicating negligible difference between the two model variants.
• Hierarchical community detection on the extracted core revealed coherent clusters with interpretable keywords, such as 'lasso', 'sparsity', and 'model selection' in cluster 1 and 2.
• The core extracted by the proposed method led to a more meaningful and interpretable hierarchical structure than all baseline methods, as validated by abstract similarity metrics.
• The method demonstrated strong consistency in core identification under mild conditions, ensuring downstream analyses remain valid once the core is correctly extracted.
• The results show that filtering non-informative periphery nodes significantly enhances the quality of hierarchical community detection in citation networks.