[Paper Review] Probabilistic End-to-end Noise Correction for Learning with Noisy Labels

Kun Yi, Jianxin Wu|arXiv (Cornell University)|Mar 19, 2019

Machine Learning and Data ClassificationComputer Science31 references67 citations

TL;DR

PENCIL trains neural networks while simultaneously updating label distributions to correct noisy labels end-to-end, without needing a clean auxiliary dataset or prior noise information.

ABSTRACT

Deep learning has achieved excellent performance in various computer vision tasks, but requires a lot of training examples with clean labels. It is easy to collect a dataset with noisy labels, but such noise makes networks overfit seriously and accuracies drop dramatically. To address this problem, we propose an end-to-end framework called PENCIL, which can update both network parameters and label estimations as label distributions. PENCIL is independent of the backbone network structure and does not need an auxiliary clean dataset or prior information about noise, thus it is more general and robust than existing methods and is easy to apply. PENCIL outperforms previous state-of-the-art methods by large margins on both synthetic and real-world datasets with different noise types and noise rates. Experiments show that PENCIL is robust on clean datasets, too.

Motivation & Objective

• Motivate robust learning under noisy labels in deep networks where clean data or noise priors are unavailable.
• Propose a probabilistic label model that can be updated during training to correct noise.
• Develop an end-to-end learning framework that updates both network parameters and per-sample label distributions.
• Demonstrate robustness and state-of-the-art performance across synthetic and real-world noisy-label datasets.

Proposed method

• Model each image label as a distribution over classes (label distributions) rather than a fixed one-hot label.
• Jointly optimize network parameters and per-sample label distributions via backpropagation.
• Use a classification loss based on a KL-divergence variant with respect to the label distributions (L_c) and explain its gradient properties to encourage effective noise correction.
• Include a compatibility loss (L_o) that ties the label distributions to the noisy labels without discarding useful information.
• Add an entropy regularization term (L_e) to prevent premature convergence to flat predictions and encourage discriminative outputs.
• Provide a three-step training workflow: backbone pre-training with high learning rate, PENCIL-end-to-end learning to correct noise, and final fine-tuning with fixed label distributions.

Experimental results

Research questions

• RQ1 Can label distributions be effectively updated end-to-end to correct noisy labels without external clean data or noise priors?
• RQ2 Does replacing fixed labels with learnable label distributions improve robustness to symmetric and asymmetric noise across datasets?
• RQ3 How does the proposed loss formulation influence gradient updates for correcting noisy labels compared to traditional KL losses?
• RQ4 Is PENCIL robust on real-world noisy data and on clean datasets?

Key findings

• PENCIL outperforms prior methods by large margins on CIFAR-10, CIFAR-100, and Clothing1M under various noise types and rates.
• The framework remains robust on clean datasets, and the learned label distributions align with true labels in practice.
• PENCIL exhibits strong robustness to moderate noise levels across both synthetic (symmetric and asymmetric) and real-world datasets.
• An extreme case with very high noise (80% symmetric) reveals a failure mode where correct labels become a minority and the method struggles, highlighting its limits.
• The approach does not require auxiliary clean data or prior knowledge about the noise transition matrix, making it broadly applicable.