[论文解读] Classification with Sparse Overlapping Groups

Nikhil Rao, Robert D. Nowak+2 more|arXiv (Cornell University)|Feb 18, 2014

Sparse and Compressive Sensing Techniques参考文献 32被引用 8

一句话总结

本文提出了稀疏重叠组（SOG）lasso，一种用于高维分类的凸优化框架，通过基于相似性的重叠组选择特征，实现结构化稀疏性。该研究首次为分类任务中的稀疏组lasso建立了模型选择误差界，推广了lasso和组lasso，并在fMRI、MEG和基因表达任务的真实与合成数据上表现出更优性能。

ABSTRACT

Classification with a sparsity constraint on the solution plays a central role in many high dimensional machine learning applications. In some cases, the features can be grouped together so that entire subsets of features can be selected or not selected. In many applications, however, this can be too restrictive. In this paper, we are interested in a less restrictive form of structured sparse feature selection: we assume that while features can be grouped according to some notion of similarity, not all features in a group need be selected for the task at hand. When the groups are comprised of disjoint sets of features, this is sometimes referred to as the "sparse group" lasso, and it allows for working with a richer class of models than traditional group lasso methods. Our framework generalizes conventional sparse group lasso further by allowing for overlapping groups, an additional flexiblity needed in many applications and one that presents further challenges. The main contribution of this paper is a new procedure called Sparse Overlapping Group (SOG) lasso, a convex optimization program that automatically selects similar features for classification in high dimensions. We establish model selection error bounds for SOGlasso classification problems under a fairly general setting. In particular, the error bounds are the first such results for classification using the sparse group lasso. Furthermore, the general SOGlasso bound specializes to results for the lasso and the group lasso, some known and some new. The SOGlasso is motivated by multi-subject fMRI studies in which functional activity is classified using brain voxels as features, source localization problems in Magnetoencephalography (MEG), and analyzing gene activation patterns in microarray data analysis. Experiments with real and synthetic data demonstrate the advantages of SOGlasso compared to the lasso and group lasso.

研究动机与目标

• 为解决传统lasso和组lasso在高维分类中的局限性，通过重叠特征组实现结构化稀疏性。
• 开发一种灵活的特征选择方法，允许组内部分选择，反映现实场景中仅部分相似特征相关的情况。
• 为使用稀疏重叠组lasso进行分类的模型选择误差提供理论保证。
• 在fMRI、MEG和基因微阵列分析等应用中展示该方法的有效性，这些应用中特征自然形成重叠组。

提出的方法

• 提出SOGlasso优化框架，一种结合l1正则化与重叠组lasso惩罚的凸规划，以在相似且重叠的特征组中鼓励稀疏性。
• 采用混合范数惩罚，对每组内系数的l1范数进行惩罚，并对组间系数的l2范数进行惩罚，实现组的局部选择。
• 利用高斯宽度和测度集中技术推导模型选择误差界，通过协方差矩阵Σ考虑特征相关性。
• 建立的误差界可退化为lasso和组lasso的已知结果，验证了该框架的通用性。
• 通过使用协方差矩阵的平方根对设计矩阵进行变换，将优化方法适配于相关特征，保持统计性质。
• 采用约束优化公式，解在按协方差矩阵平方根缩放的约束集中导出，确保对相关性的鲁棒性。

实验结果

研究问题

• RQ1 能否设计一种凸优化框架，在特征被划分为重叠集合时实现结构化稀疏特征选择？
• RQ2 在高维设置下，使用重叠组lasso进行分类时，可建立哪些理论误差界？
• RQ3 SOGlasso方法在模型选择准确性和特征恢复方面与lasso和组lasso相比如何？
• RQ4 SOGlasso框架能否推广以处理fMRI和基因表达数据等真实世界数据中的相关特征？
• RQ5 SOGlasso方法在具有自然组结构的应用中是否保持理论一致性并提升可解释性？

主要发现

• SOGlasso框架首次为使用稀疏重叠组lasso进行分类建立了模型选择误差界，为其应用提供了理论依据。
• 误差界可退化为lasso和组lasso的已知结果，证实了该框架的一致性与通用性。
• 理论分析表明，所需样本数与协方差矩阵的条件数、组数以及组数的对数成比例。
• 在真实与合成数据上的实证结果表明，SOGlasso在特征选择准确性和分类性能方面优于lasso和组lasso。
• 该方法能有效恢复重叠组结构中的相关特征，如基因通路中的基因或脑网络中的体素，提升可解释性。
• 通过使用协方差矩阵的平方根对设计矩阵进行变换，分析考虑了特征相关性，使在现实数据条件下获得更紧的误差界。