[论文解读] SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise

Feng Chen, Georgios Tzimiropoulos+1 more|arXiv (Cornell University)|Nov 22, 2021

Machine Learning and Data Classification参考文献 45被引用 9

一句话总结

本文提出SSR，一种简单而稳健的框架，用于学习未知标签噪声（LULN）情形，其中噪声类型和程度均未指定。该框架采用非参数K近邻分类器（KNN）进行干净样本筛选，利用参数化模型迭代重标注噪声样本，在CIFAR-10/100上实现合成噪声下的最先进准确率，并在WebVision、Clothing1M和ANIMAL-10N等真实世界数据集上表现优异，且无需半监督学习或预训练。

ABSTRACT

Despite the large progress in supervised learning with neural networks, there are significant challenges in obtaining high-quality, large-scale and accurately labelled datasets. In such a context, how to learn in the presence of noisy labels has received more and more attention. As a relatively complex problem, in order to achieve good results, current approaches often integrate components from several fields, such as supervised learning, semi-supervised learning, transfer learning and resulting in complicated methods. Furthermore, they often make multiple assumptions about the type of noise of the data. This affects the model robustness and limits its performance under different noise conditions. In this paper, we consider a novel problem setting, Learning with Unknown Label Noise}(LULN), that is, learning when both the degree and the type of noise are unknown. Under this setting, unlike previous methods that often introduce multiple assumptions and lead to complex solutions, we propose a simple, efficient and robust framework named Sample Selection and Relabelling(SSR), that with a minimal number of hyperparameters achieves SOTA results in various conditions. At the heart of our method is a sample selection and relabelling mechanism based on a non-parametric KNN classifier~(NPK) $g_q$ and a parametric model classifier~(PMC) $g_p$, respectively, to select the clean samples and gradually relabel the noisy samples. Without bells and whistles, such as model co-training, self-supervised pre-training and semi-supervised learning, and with robustness concerning the settings of its few hyper-parameters, our method significantly surpasses previous methods on both CIFAR10/CIFAR100 with synthetic noise and real-world noisy datasets such as WebVision, Clothing1M and ANIMAL-10N. Code is available at https://github.com/MrChenFeng/SSR_BMVC2022.

研究动机与目标

• 解决在未知标签噪声下训练深度神经网络的挑战，其中噪声类型和程度均未指定。
• 开发一种对超参数设置不敏感且不依赖复杂组件（如自监督预训练或协同训练）的方法。
• 通过结合样本筛选与迭代重标注，不显式建模噪声模式，从而提升在噪声数据集上的泛化能力与准确率。
• 在多种噪声设置（包括对称、非对称，以及开放集与闭集噪声的组合）下实现最先进性能。

提出的方法

• SSR使用非参数K近邻分类器（NPK）基于样本与其k个最近邻标签的一致性来识别干净样本。
• 使用参数化模型分类器（PMC）生成预测，并基于高置信度预测迭代重标注噪声样本。
• 该框架采用两阶段流程：首先通过NPK筛选干净样本；其次利用PMC的高置信度预测对噪声样本进行重标注。
• 可选地应用特征一致性损失，通过鼓励特征表示的一致性，提升对开放集噪声的鲁棒性。
• 该方法避免使用复杂组件，如模型协同训练、自监督预训练或半监督学习，仅依赖交叉熵损失与迭代优化。
• 超参数数量极少且具有鲁棒性，消融研究显示其在多种噪声条件下对参数值不敏感。

实验结果

研究问题

• RQ1 简单框架是否能在未知标签噪声学习中超越复杂最先进方法？
• RQ2 所提出的样本筛选与重标注机制在未知噪声类型与不同噪声水平下表现如何？
• RQ3 在真实世界噪声数据集上，缺乏半监督学习或预训练是否会影响性能？
• RQ4 该方法在不同噪声设置下对超参数选择的鲁棒性如何？
• RQ5 该框架如何处理组合式开放集与闭集噪声（现有方法常在此类设置下失效）？

主要发现

• 在CIFAR-10上，SSR在90%对称噪声下达到96.3%的top-1准确率，显著优于先前SOTA方法。
• 在CIFAR-100上，SSR在90%对称噪声下达到95.2%的准确率，较此前最佳方法提升超过1.5个百分点。
• 在具有挑战性的组合噪声设置下（0.5开放集噪声，1.0闭集噪声），SSR在CIFAR-10上达到95.7%的top-1准确率，优于EDM（94.0%）与DivideMix（91.8%）。
• 在真实世界WebVision数据集上，SSR+达到80.92%的top-1准确率，超过Co-teaching（63.58%）与DivideMix（77.32%）。
• 在ANIMAL-10N数据集上，SSR+达到88.5%的top-1准确率，优于SELFIE（79.4%）、PLC（81.8%）与NCT（83.4%）。
• 消融研究证实，SSR对超参数值不敏感，且在无需半监督学习或预训练的情况下仍保持强性能。