[Paper Review] Learning Data Manipulation for Augmentation and Weighting

Zhiting Hu, Bowen Tan+3 more|arXiv (Cornell University)|Oct 28, 2019

Machine Learning and Data Classification71 citations

TL;DR

The paper presents a unified gradient-based framework that learns data manipulation techniques (augmentation and weighting) by parameterizing a data reward and jointly optimizing manipulation and model parameters, outperforming baselines on text and image tasks in low-data and imbalanced settings.

ABSTRACT

Manipulating data, such as weighting data examples or augmenting with new instances, has been increasingly used to improve model training. Previous work has studied various rule- or learning-based approaches designed for specific types of data manipulation. In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm. Our approach builds upon a recent connection of supervised learning and reinforcement learning (RL), and adapts an off-the-shelf reward learning algorithm from RL for joint data manipulation learning and model training. Different parameterization of the "data reward" function instantiates different manipulation schemes. We showcase data augmentation that learns a text transformation network, and data weighting that dynamically adapts the data sample importance. Experiments show the resulting algorithms significantly improve the image and text classification performance in low data regime and class-imbalance problems.

Motivation & Objective

• Motivate automated data manipulation to improve learning in low-data and imbalanced settings.
• Propose a unified reward-based framework that parameterizes data manipulation.
• Demonstrate instantiations for text augmentation and data weighting.
• Show empirical improvements over strong baselines on text and image tasks.
• Highlight the flexibility to extend to other manipulation schemes.

Proposed method

• Formulate data manipulation as a parameterized reward R_phi(x,y|D) that modifies the supervised learning objective.
• Use an EM-like gradient-based reward learning to jointly update model parameters theta and manipulation parameters phi.
• Augment data by learning a text augmentation network that substitutes words conditioned on labels.
• Learn data weights by assigning learned per-sample weights phi to training examples.
• Optimize manipulation parameters on a held-out validation set to maximize final extrinsic performance.
• Provide a unified algorithm (Algorithm 1) that alternates updating theta (Eq. 7) and phi (Eq. 8).

Experimental results

Research questions

• RQ1 Can a single gradient-based framework support multiple data manipulation schemes through reward parameterization?
• RQ2 Do learned augmentation and weighting improve performance in low-data and imbalanced settings for text and image classification?
• RQ3 How does manipulation interact with large pretrained models (e.g., BERT, ResNet) in practice?
• RQ4 What are the comparative advantages of augmentation versus weighting across tasks and data regimes?

Key findings

• Data manipulation via a parameterized reward significantly improves accuracy over base models and prior manipulation methods.
• Fine-tuned text augmentation consistently beats fixed augmentation and synonym-based approaches in low-data text tasks.
• Data weighting yields improvements over baselines and outperforms a recent reweighting method, especially under severe class imbalance.
• Augmentation tends to help more in low-data scenarios, while weighting better addresses label imbalance.
• The unified framework achieves gains on SST-5, IMDB, TREC (text), and CIFAR-10 with ResNet-34 (image).