[Paper Review] FINE Samples for Learning with Noisy Labels

Taehyeon Kim, Jongwoo Ko+3 more|arXiv (Cornell University)|Feb 23, 2021

Machine Learning and Data Classification51 references38 citations

TL;DR

FINE introduces a derivative-free detector that filters label noise by examining alignment of latent representations with the first eigenvector of class-wise Gram matrices, enabling robust sample selection, SSL, and collaboration with robust losses across benchmarks.

ABSTRACT

Modern deep neural networks (DNNs) become frail when the datasets contain noisy (incorrect) class labels. Robust techniques in the presence of noisy labels can be categorized into two folds: developing noise-robust functions or using noise-cleansing methods by detecting the noisy data. Recently, noise-cleansing methods have been considered as the most competitive noisy-label learning algorithms. Despite their success, their noisy label detectors are often based on heuristics more than a theory, requiring a robust classifier to predict the noisy data with loss values. In this paper, we propose a novel detector for filtering label noise. Unlike most existing methods, we focus on each data's latent representation dynamics and measure the alignment between the latent distribution and each representation using the eigendecomposition of the data gram matrix. Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative-free simple methods having theoretical guarantees. Under our framework, we propose three applications of the FINE: sample-selection approach, semi-supervised learning approach, and collaboration with noise-robust loss functions. Experimental results show that the proposed methods consistently outperform corresponding baselines for all three applications on various benchmark datasets.

Motivation & Objective

• Motivate robust learning from datasets with noisy labels where traditional loss-based detectors fail due to classifier bias.
• Develop a noise detector that relies on latent representation geometry rather than posterior predictions.
• Provide a theoretically grounded framework with guarantees for filtering noisy instances.
• Demonstrate the detector across three LNL applications: sample selection, semi-supervised learning, and collaboration with noise-robust losses.

Proposed method

• Construct per-class Gram matrices from pre-logit representations and perform eigen decomposition to obtain the first eigenvector for each class.
• Define alignment of a data point as the squared inner product with the class’s first eigenvector, and model the alignment distribution with a Gaussian Mixture Model to separate clean vs. noisy samples.
• Filter noisy data by selecting samples whose alignment scores place them in the clean component of the GMM, without requiring noise-rate estimates.
• Provide a theoretical bound (Theorem 1) on the perturbation of the estimated clean eigenvector under label noise, linking perturbation to the noise ratio and the angle between clean/noisy class directions.
• Show scalability by approximating the eigenvector with a small subset of data, maintaining high accuracy.
• Integrate FINE into three LNL paradigms: (1) sample-selection (replace existing detectors), (2) SSL (replace loss-based filtering), (3) collaboration with noise-robust losses.

Experimental results

Research questions

• RQ1 Can latent-space eigenvector structure provide a robust, derivative-free detector for noisy labels without estimating noise rates?
• RQ2 How well does alignment to the first eigenvector separate clean from noisy samples across datasets and noise patterns?
• RQ3 Do FINE-based detectors improve performance when embedded in sample-selection, SSL, and robust-loss collaborations compared to existing detectors?
• RQ4 What theoretical guarantees can link the detector’s eigenvector alignment to clean-data identification under realistic assumptions?

Key findings

• FINE consistently outperforms competitive baselines in sample-selection tasks across CIFAR-10/100 under symmetric and asymmetric noise in multiple settings.
• Replacing loss-based filtering with FINE yields substantial gains when integrated with Co-teaching variants (F-Co-teaching) and with TopoFilter/CRUST baselines.
• In SSL, FINE-enhanced DivideMix (F-DivideMix) achieves higher test accuracy than DivideMix and is competitive with leading SSL methods under severe noise.
• FINE-guided collaboration with noise-robust losses (GCE, SCE, ELR) improves generalization in high-noise regimes.
• Experiments on Clothing1M show competitive performance, indicating real-world applicability of FINE beyond synthetic benchmarks.