[论文解读] Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition

Yifan Zhang, Bryan Hooi+2 more|arXiv (Cornell University)|Jul 20, 2021

Domain Adaptation and Few-Shot Learning被引用 55

一句话总结

SADE 从一个长尾数据集中学习多位具备多样技能的专家，并使用自监督的测试时聚合来处理未知的测试类别分布，而无需事先知识。

ABSTRACT

Existing long-tailed recognition methods, aiming to train class-balanced models from long-tailed data, generally assume the models would be evaluated on the uniform test class distribution. However, practical test class distributions often violate this assumption (e.g., being either long-tailed or even inversely long-tailed), which may lead existing methods to fail in real applications. In this paper, we study a more practical yet challenging task, called test-agnostic long-tailed recognition, where the training class distribution is long-tailed while the test class distribution is agnostic and not necessarily uniform. In addition to the issue of class imbalance, this task poses another challenge: the class distribution shift between the training and test data is unknown. To tackle this task, we propose a novel approach, called Self-supervised Aggregation of Diverse Experts, which consists of two strategies: (i) a new skill-diverse expert learning strategy that trains multiple experts from a single and stationary long-tailed dataset to separately handle different class distributions; (ii) a novel test-time expert aggregation strategy that leverages self-supervision to aggregate the learned multiple experts for handling unknown test class distributions. We theoretically show that our self-supervised strategy has a provable ability to simulate test-agnostic class distributions. Promising empirical results demonstrate the effectiveness of our method on both vanilla and test-agnostic long-tailed recognition. Code is available at \url{https://github.com/Vanint/SADE-AgnosticLT}.

研究动机与目标

• 激励在测试分布不均匀时的实际长尾识别研究。
• 在单一长尾数据集上训练多位多样化的专家。
• 提出一种自监督的测试时聚合以处理未知的测试分布。

提出的方法

• 训练三个具有不同损失的专家，以覆盖长尾、均匀和逆长尾分布（前向、均匀、后向）。
• 前向专家 E1 使用交叉熵来模拟长尾训练分布。
• 均匀专家 E2 使用平衡 Softmax 来模拟均匀分布。
• 后向专家 E3 使用逆 Softmax 损失来模拟逆长尾分布。
• 通过在未标记测试样本的两个增强视图上最大化预测稳定性来学习测试时聚合权重。
• 聚合形式为 y_hat = softmax(w1 v1 + w2 v2 + w3 v3)，其中 w 归一化使和为 1。
• 理论基础表明该目标与最大化预测分布与真实测试分布之间的互信息相关。

实验结果

研究问题

• RQ1 如何从一个单一的长尾训练集创建多位多样化的专家？
• RQ2 自监督聚合是否能在没有先验测试分布知识的情况下适应未知的测试类别分布？
• RQ3 在测试时最大化预测稳定性是否会得到有效模拟未知测试分布的权重？

主要发现

• SADE 在多个数据集上实现了强劲的原生长尾识别性能。
• 在 ImageNet-LT 上，SADE 获得 66.5（Many）、57.0（Medium）、43.5（Few）、58.8（All），超过若干基线。
• SADE 在多种设置下，SADE 在 ImageNet-LT 上优于 RIDE 和 ACE。
• 对于测试无关的长尾识别，SADE 超越依赖已知测试先验（如 LADE）的方法，并在前向、均匀和后向分布下显示出鲁棒性。
• 预测稳定性与专家强度相关，能够在测试时实现有效的自监督聚合。
• 理论分析将预测稳定性与互信息 I(Y_hat; Y) 和测试分布对齐联系起来。