[Paper Review] Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels

Brandon Smart, Gustavo Carneiro|arXiv (Cornell University)|Oct 17, 2022

Water Systems and Optimization4 citations

TL;DR

This paper proposes a three-stage training algorithm that learns the relationship between images, clean labels, and noisy labels without requiring a pre-labeled clean dataset. By using self-supervised pretraining, early-stopping to identify a confident clean set, and semi-supervised learning to relabel the remainder, the method achieves state-of-the-art performance on asymmetric and instance-dependent label noise benchmarks using a single, simple model architecture.

ABSTRACT

Many state-of-the-art noisy-label learning methods rely on learning mechanisms that estimate the samples' clean labels during training and discard their original noisy labels. However, this approach prevents the learning of the relationship between images, noisy labels and clean labels, which has been shown to be useful when dealing with instance-dependent label noise problems. Furthermore, methods that do aim to learn this relationship require cleanly annotated subsets of data, as well as distillation or multi-faceted models for training. In this paper, we propose a new training algorithm that relies on a simple model to learn the relationship between clean and noisy labels without the need for a cleanly labelled subset of data. Our algorithm follows a 3-stage process, namely: 1) self-supervised pre-training followed by an early-stopping training of the classifier to confidently predict clean labels for a subset of the training set; 2) use the clean set from stage (1) to bootstrap the relationship between images, noisy labels and clean labels, which we exploit for effective relabelling of the remaining training set using semi-supervised learning; and 3) supervised training of the classifier with all relabelled samples from stage (2). By learning this relationship, we achieve state-of-the-art performance in asymmetric and instance-dependent label noise problems.

Motivation & Objective

• Address the challenge of instance-dependent label noise (IDN) in real-world datasets where clean labels are unavailable.
• Learn the relationship between images, noisy labels, and clean labels without relying on a pre-labeled clean subset.
• Eliminate the need for model distillation by introducing a 'label dropping' strategy for end-to-end inference.
• Achieve state-of-the-art performance on asymmetric and instance-dependent label noise benchmarks using a simple, single-model architecture.

Proposed method

• Perform self-supervised pretraining on the full training set using contrastive learning (e.g., SimCLR) to learn robust image representations.
• Train the classifier with 'null' labels to predict noisy labels, using early stopping to identify a subset of high-confidence predictions as a pseudo-clean set.
• Apply noise-transition sample balancing during bootstrapping to prevent bias toward already-correct samples and ensure diverse noise transitions are represented.
• Use the identified clean set in a semi-supervised learning stage to explicitly learn the relationship between images, noisy labels, and clean labels, enabling effective relabeling of the remaining data.
• Perform final supervised training on the entire dataset using the relabeled samples, with MixUp and strong data augmentations to improve generalization.
• Introduce a 'label dropping' strategy where noisy labels are randomly replaced with 'null' labels during training, eliminating the need for distillation and enabling inference on both labeled and unlabeled samples.

Figure 1: On the left, we show a ‘normal’ deep neural network model used for noisy label learning tasks. On the right, we present a ‘modified’ model that can learn the relationship between images $\mathbf{x}$ , noisy labels $\tilde{\mathbf{y}}$ and clean labels $\mathbf{y}$ , similar to those used b

Experimental results

Research questions

• RQ1 Can a model learn the relationship between images, noisy labels, and clean labels without access to a pre-labeled clean dataset?
• RQ2 Does a three-stage bootstrapping process combining self-supervision, early-stopping, and semi-supervised learning improve relabeling accuracy in instance-dependent noise settings?
• RQ3 Can noise-transition balancing during bootstrapping prevent bias in the selected clean set and improve downstream relabeling performance?
• RQ4 Does the proposed method achieve state-of-the-art performance on asymmetric and instance-dependent label noise benchmarks using a single, simple model architecture?
• RQ5 Can the 'label dropping' strategy eliminate the need for model distillation while maintaining performance on both noisy and clean samples?

Key findings

• The method achieves state-of-the-art accuracy of 95.85% on CIFAR-10 with 40% asymmetric label noise, outperforming existing methods.
• After the bootstrapping stage, the model achieves 91.41% accuracy, which improves to 94.98% after semi-supervised learning and 95.85% after final training with MixUp and strong augmentations.
• Self-supervised pretraining reduces the number of errors in the bootstrapped clean set from 369 (without pretraining) to 21 (with pretraining), significantly improving clean set quality.
• Using strong augmentations during bootstrapping training reduces clean set errors to 31, compared to 579 with no augmentations, demonstrating the regularization effect of strong augmentations.
• The 'label dropping' strategy enables direct inference on both noisy and clean samples without distillation, simplifying deployment and model serving.
• The method achieves SOTA results on instance-dependent and asymmetric label noise benchmarks using a single-model architecture, unlike prior SOTA methods that require complex two-model setups (e.g., DivideMix).

Figure 2: Our proposed algorithm. The bootstrapping stage consists of self-supervised pre-training, followed by early-stopping classification training to identify a small set of confidently relabelled samples (i.e., clean labelled samples). This set is then used to learn the relationship between ima