[论文解读] BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition

Boyan Zhou, Quan Cui+2 more|arXiv (Cornell University)|Dec 5, 2019

Domain Adaptation and Few-Shot Learning参考文献 24被引用 79

一句话总结

BBN 提出了一种双支路网络与累积学习策略，旨在共同提升长尾视觉识别的表征与分类器学习，在若干基准测试中达到最先进的性能。

ABSTRACT

Our work focuses on tackling the challenging but natural visual recognition task of long-tailed data distribution (i.e., a few classes occupy most of the data, while most classes have rarely few samples). In the literature, class re-balancing strategies (e.g., re-weighting and re-sampling) are the prominent and effective methods proposed to alleviate the extreme imbalance for dealing with long-tailed problems. In this paper, we firstly discover that these re-balancing methods achieving satisfactory recognition accuracy owe to that they could significantly promote the classifier learning of deep networks. However, at the same time, they will unexpectedly damage the representative ability of the learned deep features to some extent. Therefore, we propose a unified Bilateral-Branch Network (BBN) to take care of both representation learning and classifier learning simultaneously, where each branch does perform its own duty separately. In particular, our BBN model is further equipped with a novel cumulative learning strategy, which is designed to first learn the universal patterns and then pay attention to the tail data gradually. Extensive experiments on four benchmark datasets, including the large-scale iNaturalist ones, justify that the proposed BBN can significantly outperform state-of-the-art methods. Furthermore, validation experiments can demonstrate both our preliminary discovery and effectiveness of tailored designs in BBN for long-tailed problems. Our method won the first place in the iNaturalist 2019 large scale species classification competition, and our code is open-source and available at https://github.com/Megvii-Nanjing/BBN.

研究动机与目标

• 研究在长尾分布下，类别再平衡如何同时影响表征与分类器学习。
• 提出一个统一的双支路网络（BBN），以联合优化表征与分类器学习。
• 开发累积学习策略，在训练中将重点从普遍特征转移到尾部数据。
• 在多个长尾基准测试（包括大规模 iNaturalist）上证明 BBN 的有效性。

提出的方法

• 两分支架构，权重共享：传统学习分支（均匀采样器）用于普遍表征，和再平衡分支（反向采样器）用于尾部聚焦的分类器学习。
• 通过自适应参数 alpha 对分支输出进行自适应融合，权衡分支贡献。
• 累积学习策略，其中 alpha 作为训练 epoch 的函数以逐步强调尾部数据。
• 端到端训练，采用加权交叉熵损失，结合两个分支的预测。
• 推理阶段对两个分支的贡献进行平均，最终预测的 alpha 固定为 0.5 以实现平衡。

实验结果

研究问题

• RQ1 长尾数据中类别再平衡策略如何同时影响特征表示与分类器学习？
• RQ2 一种分别处理表示与分类器学习的双支路设计，是否能优于单分支或两阶段方法？
• RQ3 将焦点从普遍特征转移到尾部数据的累积学习方案，是否能获得更强的长尾识别性能？
• RQ4 在两个分支之间共享骨干权重是否有助于效率和性能？

主要发现

• BBN 在长尾 CIFAR-10/100 上的对比方法持续带来改进，跨不平衡因子。
• BBN 在大型 iNaturalist 数据集上也优于基线，超过强大的两阶段微调方法。
• 为再平衡分支采用反向采样器，在尾部类别表现上优于均匀或完全平衡的采样器。
• 针对 alpha 的抛物线衰减自适应器在测试策略中给出最佳结果。
• 来自传统分支的特征表征仍具竞争力，而再平衡分支有效建模尾部数据，结合权重实现分类器行为的平衡。