[Paper Review] Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels

Songhua Wu, Xiaobo Xia+6 more|arXiv (Cornell University)|Jun 14, 2020

Machine Learning and Data ClassificationComputer Science20 citations

TL;DR

This paper proposes Class2Simi, a novel noise reduction framework that transforms noisy class labels into noisy similarity labels (indicating whether pairs of samples share the same class), thereby lowering the effective noise rate. By training deep networks on these pairwise similarity labels after pretraining on noisy pointwise labels, the method achieves state-of-the-art robustness to label noise, especially under high noise rates and estimation errors in transition matrices.

ABSTRACT

Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in pointwise manners. Meanwhile, pairwise manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner mitigate label noise? To give an affirmative answer, in this paper, we propose a framework called Class2Simi: it transforms data points with noisy class labels to data pairs with noisy similarity labels, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the reduction of the noise rate is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the clean class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is computationally efficient because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.

Motivation & Objective

• Address the challenge of training deep neural networks under high label noise in large-scale datasets.
• Investigate whether pairwise learning, which exploits relationships between data points, can inherently mitigate label noise compared to traditional pointwise methods.
• Develop a framework that transforms noisy class labels into noisy similarity labels to reduce the effective noise rate.
• Establish a theoretical and practical connection between noisy similarity posteriors and clean class posteriors for robust model training.
• Demonstrate that the proposed method is robust to errors in estimating the class transition matrix, a common challenge in noisy label learning.

Proposed method

• Transform each training sample with a noisy class label into data pairs, where each pair is assigned a noisy similarity label indicating whether the two samples share the same class.
• Theoretically prove that the noise rate of similarity labels is lower than that of class labels, using a transition matrix formalism (Theorem 2), due to the higher tolerance of similarity labels to individual label errors.
• Pretrain a deep neural network on the original noisy pointwise data using standard cross-entropy loss to preserve semantic class information before switching to pairwise learning.
• Fine-tune the model using a pairwise loss function (e.g., Forward or Reweight) that corrects for noisy similarity labels, leveraging the clean class posterior via inner product approximation.
• Use the similarity transition matrix derived from the class transition matrix to model noise in pairwise supervision, ensuring theoretical robustness.
• Implement the transformation on-the-fly during mini-batch training, minimizing computational overhead while enabling efficient pairwise learning.

Experimental results

Research questions

• RQ1 Can transforming pointwise class labels into pairwise similarity labels reduce the effective noise rate in training data?
• RQ2 Does learning from noisy similarity labels lead to better generalization than learning from noisy class labels in deep neural networks?
• RQ3 How robust is the proposed method to errors in estimating the class transition matrix, a key component in many noisy label learning approaches?
• RQ4 Can pretraining on noisy pointwise data preserve sufficient semantic information to enable effective fine-tuning on noisy pairwise labels?
• RQ5 Does the pairwise learning framework outperform existing state-of-the-art pointwise methods under symmetric and asymmetric label noise settings?

Key findings

• Class2Simi reduces the effective noise rate from 0.5 (on class labels) to 0.25 (on similarity labels), as demonstrated in Figure 1, due to the higher tolerance of similarity labels to individual label errors.
• On CIFAR-10 and CIFAR-100 with a 60% noise rate, Class2Simi improves accuracy by approximately 5 and 10 points, respectively, over baseline pointwise methods.
• F-Class2Simi achieves 75.41% accuracy on Clothing1M*, outperforming Co-teaching (74.70%) and S2E (72.30%), demonstrating strong performance on real-world noisy data.
• R-Class2Simi achieves 75.76% accuracy on Clothing1M*, further improving over F-Class2Simi, indicating the effectiveness of the reweighting variant.
• The method remains robust under perturbations to the ground-truth transition matrix: while Forward accuracy drops sharply with noise, F-Class2Simi shows only minor fluctuations, proving robustness.
• On clean datasets (MNIST, CIFAR10, CIFAR100, News20), the similarity loss provides no significant accuracy gain over standard cross-entropy, confirming that improvements in noisy settings stem from noise reduction, not loss design.