[论文解读] Descriptive vs. inferential community detection: pitfalls, myths and half-truths.

Tiago P. Peixoto|arXiv (Cornell University)|Nov 30, 2021

Complex Network Analysis Techniques参考文献 103被引用 4

一句话总结

本文区分了描述性社区检测与推断性社区检测方法，认为基于生成模型的推断性方法在回答关于网络结构的科学问题方面更优。它表明，将描述性方法用于推断性目标会导致误导性结果，原因在于未能将信号与随机性分离。

ABSTRACT

Community detection is one of the most important methodological fields of network science, and one which has attracted a significant amount of attention over the past decades. This area deals with the automated division of a network into fundamental building blocks, with the objective of providing a summary of its large-scale structure. Despite its importance and widespread adoption, there is a noticeable gap between what is considered the state-of-the-art and the methods that are actually used in practice in a variety of fields. Here we attempt to address this discrepancy by dividing existing methods according to whether they have a or an goal. While descriptive methods find patterns in networks based on intuitive notions of community structure, inferential methods articulate a precise generative model, and attempt to fit it to data. In this way, they are able to provide insights into the mechanisms of network formation, and separate structure from randomness in a manner supported by statistical evidence. We review how employing descriptive methods with inferential aims is riddled with pitfalls and misleading answers, and thus should be in general avoided. We argue that inferential methods are more typically aligned with clearer scientific questions, yield more robust results, and should be in general preferred. We attempt to dispel some myths and half-truths often believed when community detection is employed in practice, in an effort to improve both the use of such methods as well as the interpretation of their results.

研究动机与目标

• 阐明描述性与推断性社区检测方法之间的根本区别。
• 识别并批判在意图实现推断性目标时广泛存在的对描述性方法的误用。
• 论证推断性方法通过建模网络形成机制，能更好地支持科学推断。
• 消除在实际研究应用中关于社区检测的常见误解与谬误。
• 推动采用推断性方法，以实现更可靠且可解释的网络分析。

提出的方法

• 将社区检测方法分类为描述性（基于直观结构的模式发现）和推断性（基于生成模型）两类。
• 使用统计推断将生成模型拟合到网络数据，实现假设检验与随机性分离。
• 应用正式的统计框架，评估所检测到的社区是否反映真实结构而非随机波动。
• 对比描述性方法（如模块度最大化）与推断性方法（如随机块模型）的结果。
• 强调模型选择与拟合优度评估，以验证推断出的社区结构。
• 强调模型假设的重要性及其与科学问题的一致性。

实验结果

研究问题

• RQ1 为何描述性社区检测方法在用于推断性目的时常失效？
• RQ2 依赖描述性方法进行网络科学推断时，其关键统计缺陷是什么？
• RQ3 生成模型如何提升社区检测结果的可靠性和可解释性？
• RQ4 在实践中，关于社区检测的哪些误解和谬误被普遍传播？
• RQ5 推断性方法在多大程度上能更好地区分网络中的结构信号与随机噪声？

主要发现

• 描述性方法常产生在统计上无法与随机波动区分的社区，尤其是在稀疏网络中。
• 将描述性方法用于推断性目标会导致对检测到的社区产生虚假的自信，原因在于缺乏统计验证。
• 推断性方法（如随机块模型）提供了一种系统性方法，用以评估社区结构是否具有统计显著性。
• 本文表明，许多广泛使用的描述性方法（如模块度优化）易受退化问题影响，即使在随机网络中也会检测到社区。
• 推断性方法使研究人员能够检验关于网络形成机制的假设，而不仅限于总结观测到的模式。
• 研究表明，推断性方法在不同网络类型和条件下均能产生更具可重复性与鲁棒性的结果。