[Paper Review] Learning to Reweight Examples for Robust Deep Learning

Mengye Ren, Wenyuan Zeng+2 more|arXiv (Cornell University)|Mar 24, 2018

Machine Learning and Data Classification40 references579 citations

TL;DR

The paper introduces an online meta-learning method that learns to reweight training examples using a small clean validation set, improving robustness to class imbalance and noisy labels without extra hyperparameter tuning.

ABSTRACT

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these problems, but they require careful tuning of additional hyperparameters, such as example mining schedules and regularization hyperparameters. In contrast to past reweighting methods, which typically consist of functions of the cost value of each example, in this work we propose a novel meta-learning algorithm that learns to assign weights to training examples based on their gradient directions. To determine the example weights, our method performs a meta gradient descent step on the current mini-batch example weights (which are initialized from zero) to minimize the loss on a clean unbiased validation set. Our proposed method can be easily implemented on any type of deep network, does not require any additional hyperparameter tuning, and achieves impressive performance on class imbalance and corrupted label problems where only a small amount of clean validation data is available.

Motivation & Objective

• Motivate training set bias as a problem for deep networks and seek robust solutions.
• Propose a meta-learning framework that learns example weights to minimize validation loss.
• Develop an online, no-extra-hyperparameters reweighting algorithm compatible with any deep network.
• Demonstrate robustness improvements on class imbalance and noisy-label benchmarks with limited clean validation data.

Proposed method

• Formulate a two-level objective where training uses weights w for losses f_i, guided by validation loss on a small clean set.
• Compute the weights by a one-step meta-gradient on the validation loss, yielding nonnegative normalized weights.
• Implement the approach via automatic differentiation with a backward-on-backward pass to obtain gradients with respect to per-example weights.
• Normalize per-batch weights so that they sum to one, ensuring a stable effective learning rate.
• Provide convergence guarantees showing the method converges to a critical point of the validation loss under mild assumptions.
• Demonstrate applicability to any deep architecture (MLP/CNN) and discuss computational overhead (~3x) due to extra forward/backward passes.

Experimental results

Research questions

• RQ1 Can online gradient-based reweighting of training examples, guided by a small clean validation set, improve robustness to label noise and class imbalance?
• RQ2 Does the proposed meta-learning reweighting algorithm converge to the validation objective, and what is its convergence behavior?
• RQ3 How does the method perform on standard benchmarks with imbalanced data and noisy labels compared to existing reweighting and regularization techniques?
• RQ4 Is the approach practical for common architectures (MLP/CNN) without additional hyperparameter tuning?
• RQ5 What is the impact of the size of the clean validation set on performance and regularization?

Key findings

• The method yields robustness gains on both class imbalance and noisy-label problems with limited clean validation data.
• In MNIST 4-9 imbalance experiments, the method significantly outperforms baselines across imbalance ratios, including extreme 200:1.
• On CIFAR with UniformFlip and BackgroundFlip noise, the method achieves top performance relative to baselines and state-of-the-art competitors, especially with any available clean validation data.
• The algorithm tends to down-weight noisy or conflicting examples while up-weighting helpful ones, as shown by example weight distributions.
• Convergence analysis shows the method decreases the validation loss and achieves an O(1/ε^2) rate under reasonable conditions.