[论文解读] Fast Multi-Instance Multi-Label Learning

Sheng-Jun Huang, Zhi‐Hua Zhou|arXiv (Cornell University)|Oct 8, 2013

Text and Document Classification Technologies参考文献 22被引用 6

一句话总结

本文提出 MIMLfast，一种快速且高效的多实例多标签学习方法，通过为所有标签共享一个低维空间，并利用随机梯度下降训练特定标签的线性模型以优化排序损失。该方法在保持 SOTA 性能的同时，比现有方法快超过 100 倍，尤其在包含最多 180,000 个实例的大规模数据集上表现优异。

ABSTRACT

In many real-world tasks, particularly those involving data objects with complicated semantics such as images and texts, one object can be represented by multiple instances and simultaneously be associated with multiple labels. Such tasks can be formulated as multi-instance multi-label learning (MIML) problems, and have been extensively studied during the past few years. Existing MIML approaches have been found useful in many applications; however, most of them can only handle moderate-sized data. To efficiently handle large data sets, in this paper we propose the MIMLfast approach, which first constructs a low-dimensional subspace shared by all labels, and then trains label specific linear models to optimize approximated ranking loss via stochastic gradient descent. Although the MIML problem is complicated, MIMLfast is able to achieve excellent performance by exploiting label relations with shared space and discovering sub-concepts for complicated labels. Experiments show that the performance of MIMLfast is highly competitive to state-of-the-art techniques, whereas its time cost is much less; particularly, on a data set with 20K bags and 180K instances, MIMLfast is more than 100 times faster than existing MIML approaches. On a larger data set where none of existing approaches can return results in 24 hours, MIMLfast takes only 12 minutes. Moreover, our approach is able to identify the most representative instance for each label, and thus providing a chance to understand the relation between input patterns and output label semantics.

研究动机与目标

• 解决现有多实例多标签学习（MIML）方法在大规模数据上计算成本过高的问题。
• 通过利用标签关系建立共享低维空间，在不牺牲性能的前提下提升效率。
• 实现每类关键代表性实例的识别，以增强模型可解释性。
• 通过监督建模而非启发式聚类，发现复杂标签的子概念。

提出的方法

• MIMLfast 学习一个共享的线性变换矩阵 $W_0$，将原始特征映射到低维共享空间。
• 在共享空间上训练特定标签的权重向量 $\mathbf{w}_l$，以预测标签相关性。
• 使用随机梯度下降（SGD）通过采样三元组（包，相关标签，无关标签）来优化近似排序损失。
• 模型将每类预测分数最高的实例识别为该类的代表性实例。
• 通过分析共享空间中高分实例的分布，发现复杂标签的子概念。
• 联合优化共享空间和特定标签模型，以利用标签相关性。

实验结果

研究问题

• RQ1 是否能够通过共享低维空间的线性模型在多实例多标签学习中实现具有竞争力的性能，同时显著快于现有方法？
• RQ2 与独立标签建模相比，共享空间在提升泛化能力和效率方面有多有效？
• RQ3 所提出方法能否识别每类的关键代表性实例，从而实现标签语义的可解释性？
• RQ4 对复杂标签发现子概念是否能提升模型性能并提供有意义的语义洞察？

主要发现

• 在包含 20,000 个包和 180,000 个实例的数据集中，MIMLfast 的速度比现有 MIML 方法快超过 100 倍。
• 在先前方法无法在 24 小时内完成的更大规模数据集上，MIMLfast 仅用 12 分钟即获得结果。
• MIMLfast 表现具有竞争力，在 ImageCLEF 数据集上的平均精度为 .972 ± .010，在 Scene 数据集上为 .770 ± .015。
• 无共享空间的变体（V1）性能显著差于 MIMLfast，证实了共享表示学习的优势。
• 虚拟标签变体（V2）在大多数指标上与 MIMLfast 表现相当，表明虚拟标签能合理区分相关与无关标签。
• 该方法成功识别出复杂标签的子概念，例如图 6 所示的标签 'sea' 的不同视觉模式。