[论文解读] Phase Transitions in Unsupervised Feature Selection

Jonathan Fiorentino, Michele Monti+4 more|arXiv (Cornell University)|Jan 31, 2026

Machine Learning in Bioinformatics被引用 0

一句话总结

论文分析了基于 Differentiable Information Imbalance (DII) 的无监督特征选择管道，应用于蛋白质特征集，揭示了依赖于特征类型和相关结构的类似相变的行为，并将临界特征数与监督分类性能联系起来。

ABSTRACT

Identifying minimal and informative feature sets is a central challenge in data analysis, particularly when few data points are available. Here we present a theoretical analysis of an unsupervised feature selection pipeline based on the Differentiable Information Imbalance (DII). We consider the specific case of structural and physico-chemical features describing a set of proteins. We show that if one considers the features as coordinates of a (hypothetical) statistical physics model, this model undergoes a phase transition as a function of the number of retained features. For physico-chemical descriptors, this transition is between a glass-like phase when the features are few and a liquid-like phase. The glass-like phase exhibits bimodal order-parameter distributions and Binder cumulant minima. In contrast, for structural descriptors the transition is less sharp. Remarkably, for physico-chemical descriptors the critical number of features identified from the DII coincides with the saturation of downstream binary classification performance. These results provide a principled, unsupervised criterion for minimal feature sets in protein classification and reveal distinct mechanisms of criticality across different feature types.

研究动机与目标

• 在标注数据稀缺时激励无监督特征选择。
• 研究 DII 如何作为在选择信息性特征子集时的序参量。
• 表征特征集结构（物理化学 vs 结构）对信息景观的影响。
• 将无监督的临界特征数与下游二分类性能相关联。

提出的方法

• 将 DII 定义并计算为特征子集的无监督序参量。
• 对物理化学和结构特征集应用基于 DII 的向后特征消除。
• 分析随机子样本的 DII 分布以研究景观的崎岖程度。
• 使用 Binder 势能量分析识别指示转变点的临界特征数。
• 训练分类器（MLP）以将特征计数与二分类性能 AUROC 相关联。

Figure 1: Criticality in the Differentiable Information Imbalance during feature elimination. (A,B) Average DII versus the number of non-zero features $F$ for the LLPS dataset, for physico-chemical (A) and structural (B) features. (C,D) Heatmaps of the log-transformed probability density of the DII

实验结果

研究问题

• RQ1 当保留的特征数量增加时，DII 是否表现出类似相变的行为？
• RQ2 特征集的性质（物理化学 vs 结构）如何影响转变类型（玻璃态般 vs 交叉）？
• RQ3 无监督的临界特征数是否与下游分类性能的饱和点相关？
• RQ4 特征集中的相关性和方差异质性如何驱动信息景观？

主要发现

• 物理化学特征显示出玻璃般的转变，DII 景观呈现双峰分布且 Binder 均量最小值。
• 结构特征显示出较弱、较平滑的转变或交叉，DII 分布为单峰。
• 相关性结构驱动物理化学特征的转变，而方差异质性驱动结构特征的转变。
• 物理化学描述符的临界特征数与使用 DII 选择特征时二分类性能的饱和点一致。
• 从高层次看，信息性特征在约束下表现为相互作用的自由度，将临界性与蛋白分类中的泛化能力联系起来。

Figure 2: Binder cumulant analysis reveals a glass-like phase transition for physico-chemical features. (A,C) Binder cumulant $U(F)$ as a function of the number of non-zero features $F$ for physico-chemical (A) and structural (C) descriptors, for the LLPS dataset. (B,D) Extrapolation of $F_{min}$ (p