[论文解读] Compositional Few-Shot Recognition with Primitive Discovery and Enhancing

Yixiong Zou, Shanghang Zhang+4 more|arXiv (Cornell University)|May 12, 2020

Domain Adaptation and Few-Shot Learning参考文献 64被引用 5

一句话总结

该论文提出了一种组合式 few-shot 学习框架，通过自监督的物体分割顺序预测从已知类别中发现视觉基元，并通过软组合机制增强重要基元。通过联合训练基元发现与增强，该方法在 few-shot 图像与视频识别基准上实现了最先进性能，同时通过稀疏性和特征可视化提升了可解释性。

ABSTRACT

Few-shot learning (FSL) aims at recognizing novel classes given only few training samples, which still remains a great challenge for deep learning. However, humans can easily recognize novel classes with only few samples. A key component of such ability is the compositional recognition that human can perform, which has been well studied in cognitive science but is not well explored in FSL. Inspired by such capability of humans, to imitate humans' ability of learning visual primitives and composing primitives to recognize novel classes, we propose an approach to FSL to learn a feature representation composed of important primitives, which is jointly trained with two parts, i.e. primitive discovery and primitive enhancing. In primitive discovery, we focus on learning primitives related to object parts by self-supervision from the order of image splits, avoiding extra laborious annotations and alleviating the effect of semantic gaps. In primitive enhancing, inspired by current studies on the interpretability of deep networks, we provide our composition view for the FSL baseline model. To modify this model for effective composition, inspired by both mathematical deduction and biological studies (the Hebbian Learning rule and the Winner-Take-All mechanism), we propose a soft composition mechanism by enlarging the activation of important primitives while reducing that of others, so as to enhance the influence of important primitives and better utilize these primitives to compose novel classes. Extensive experiments on public benchmarks are conducted on both the few-shot image classification and video recognition tasks. Our method achieves the state-of-the-art performance on all these datasets and shows better interpretability.

研究动机与目标

• 为解决 few-shot 视觉识别中的挑战，即模型必须在仅有少量样本的情况下泛化到新类别。
• 通过从已知类别中学习可重用的视觉基元，模仿人类的组合式识别。
• 减少对昂贵标注的依赖，并缓解已知类别与新类别之间的语义鸿沟。
• 通过识别并增强特征表示中最具影响力的基元，提升模型可解释性。
• 通过联合优化基元发现与增强，实现 superior 的 few-shot 识别性能。

提出的方法

• 通过图像分割顺序预测的自监督方法实现基元发现：图像被水平和垂直分割，模型预测分割顺序的排列，从而鼓励将物体部分识别为基元。
• 模型通过分割顺序提供的结构化监督，无须显式标注即可学习与部件相关的基元，减少语义鸿沟的影响。
• 引入一种软组合机制，通过增强高激活基元并抑制不相关基元，提升其重要性，受 Hebbian 学习与 winner-take-all 动力学启发。
• 该方法使用可学习的权重矩阵 W 来衡量基元的重要性，并引入稀疏性损失以促进关键基元的选择性激活。
• 应用熵正则化（ER）损失以放大高激活基元的影响，提升其在分类中的贡献。
• 框架与余弦分类器联合训练，并采用端到端优化，实现从学习到的基元中有效组合新类别。

实验结果

研究问题

• RQ1 仅通过图像分割顺序的自监督学习，能否在无人工标注的情况下有效发现与部件相关的视觉基元？
• RQ2 在特征组合过程中，如何动态增强重要视觉基元以提升 few-shot 识别性能？
• RQ3 与标准 FSL 基线相比，所提出的软组合机制在新类别泛化方面改善程度如何？
• RQ4 所学习的基元是否能在已知类别与新类别之间实现有意义的解释与共享？
• RQ5 基元发现与增强的联合训练是否能带来在 few-shot 图像与视频识别中更好的性能与可解释性？

主要发现

• 所提方法在所有评估的 few-shot 图像与视频识别基准上均实现了最先进性能，优于现有 FSL 方法。
• 消融实验表明，每个组件——基元发现、稀疏性损失与 ER 损失——均对模型性能有积极贡献。
• 仅需 15 个特征通道即可恢复 95% 的新类别分类准确率，证明了所学基元的稀疏性与高效性。
• 权重矩阵 W 中前 5 个通道贡献了已知类别分类性能的大部分，验证了 W 作为基元重要性可靠度量的有效性。
• 可视化结果表明，已知类别与新类别之间共享的基元对应于可解释的物体部件（如腿、颈部、躯干），证实了组合泛化能力。
• 该方法表现出改进的可解释性，显著激活的基元与人类可理解的物体组件一致，例如新类别中的狗形胸部或钓鱼竿形颈部。