[Paper Review] Community models for networks observed through edge nominations

Tianxi Li, Elizaveta Levina+1 more|arXiv (Cornell University)|Aug 9, 2020

Complex Network Analysis Techniques41 references4 citations

TL;DR

This paper proposes a general model for community detection in networks collected via edge nomination—where nodes report their connections—accounting for sampling bias and missing data. It introduces the nomination stochastic block model (NSBM), enabling efficient spectral clustering and method-of-moments estimation with theoretical consistency guarantees, validated through simulations and a faculty hiring network revealing a meaningful hierarchy among U.S. business schools.

ABSTRACT

Communities are a common and widely studied structure in networks, typically under the assumption that the network is fully and correctly observed. In practice, network data are often collected by querying nodes about their connections. In some settings, all edges of a sampled node will be recorded, and in others, a node may be asked to name its connections. These sampling mechanisms introduce noise and bias which can obscure the community structure and invalidate assumptions underlying standard community detection methods. We propose a general model for a class of network sampling mechanisms based on recording edges via querying nodes, designed to improve community detection for network data collected in this fashion. We model edge sampling probabilities as a function of both individual preferences and community parameters, and show community detection can be performed by spectral clustering under this general class of models. We also propose, as a special case of the general framework, a parametric model for directed networks we call the nomination stochastic block model, which allows for meaningful parameter interpretations and can be fitted by the method of moments. Both spectral clustering and the method of moments in this case are computationally efficient and come with theoretical guarantees of consistency. We evaluate the proposed model in simulation studies on both unweighted and weighted networks and apply it to a faculty hiring dataset, discovering a meaningful hierarchy of communities among US business schools.

Motivation & Objective

• To address community detection in networks where edge data are collected via node nominations, which introduces bias and missingness.
• To develop a statistically principled model that accounts for both individual nomination preferences and community structure.
• To provide computationally efficient and theoretically consistent methods for community detection under this sampling mechanism.
• To demonstrate the method's superiority over standard approaches on simulated and real-world networks, including a faculty hiring network.
• To reveal interpretable community structures in real data by correcting for nomination bias.

Proposed method

• Proposes a general model where edge nomination probabilities depend on both node-specific preferences and community memberships.
• Introduces the nomination stochastic block model (NSBM) as a parametric special case with interpretable parameters for directed networks.
• Uses spectral clustering on a corrected adjacency matrix to detect communities, with theoretical consistency guarantees.
• Applies the method of moments to estimate NSBM parameters efficiently, avoiding MCMC or variational inference.
• Models both unweighted and weighted networks, allowing for heterogeneous nomination behaviors.
• Employs a transformation of the observed nomination matrix to recover latent community structure under the assumed model.

Experimental results

Research questions

• RQ1 How can community detection be improved when network data are collected through node edge nominations rather than full observation?
• RQ2 What is the impact of nomination bias—where missing edges depend on node and community characteristics—on standard community detection methods?
• RQ3 Can a parametric model be developed that captures both nomination behavior and community structure with interpretable parameters?
• RQ4 How do spectral clustering and method-of-moments estimation perform under this model in terms of consistency and computational efficiency?
• RQ5 What hierarchical community structure emerges when applying the model to real-world data like faculty hiring networks?

Key findings

• The proposed NSBM model successfully recovers a meaningful hierarchical structure in the U.S. business school hiring network, consistent with prior observations by Clauset et al. (2015).
• Spectral clustering under the NSBM framework outperforms standard symmetric spectral clustering on the undirected version, as shown by significant differences in community average rankings (e.g., 19.2 vs. 55.1 in US News 2012).
• The method of moments provides consistent parameter estimates for the NSBM with computational efficiency, avoiding costly MCMC or variational inference.
• Simulations on unweighted and weighted networks confirm the method’s robustness and improved accuracy under various nomination mechanisms.
• Community detection using the corrected model reveals interpretable groupings, while standard methods produce arbitrary clusters with similar average rankings.
• The framework demonstrates that ignoring nomination mechanisms leads to a substantial drop in accuracy and loss of efficiency in community detection.