[Paper Review] A Survey on Data Augmentation in Large Model Era

Yue Zhou, Chenlu Guo+3 more|arXiv (Cornell University)|Jan 27, 2024

Big Data Technologies and Applications7 citations

TL;DR

This paper provides an exhaustive survey of data augmentation methods that leverage large models (LLMs and diffusion models), categorized by image, text, and paired data, plus data post-processing and applications across NLP, CV, and audio. It also discusses challenges, future directions, and maintains open-source resources.

ABSTRACT

Large models, encompassing large language and diffusion models, have shown exceptional promise in approximating human-level intelligence, garnering significant interest from both academic and industrial spheres. However, the training of these large models necessitates vast quantities of high-quality data, and with continuous updates to these models, the existing reservoir of high-quality data may soon be depleted. This challenge has catalyzed a surge in research focused on data augmentation methods. Leveraging large models, these data augmentation techniques have outperformed traditional approaches. This paper offers an exhaustive review of large model-driven data augmentation methods, adopting a comprehensive perspective. We begin by establishing a classification of relevant studies into three main categories: image augmentation, text augmentation, and paired data augmentation. Following this, we delve into various data post-processing techniques pertinent to large model-based data augmentation. Our discussion then expands to encompass the array of applications for these data augmentation methods within natural language processing, computer vision, and audio signal processing. We proceed to evaluate the successes and limitations of large model-based data augmentation across different scenarios. Concluding our review, we highlight prospective challenges and avenues for future exploration in the field of data augmentation. Our objective is to furnish researchers with critical insights, ultimately contributing to the advancement of more sophisticated large models. We consistently maintain the related open-source materials at: https://github.com/MLGroup-JLU/LLM-data-aug-survey.

Motivation & Objective

• Classify large-model-driven data augmentation studies into image, text, and paired data categories.
• Review data post-processing techniques used with large models (top-k, model-based, score-based, cluster-based).
• Survey applications of data augmentation with LLMs and diffusion models in NLP, CV, and audio.
• Identify successes, limitations, and future challenges to guide future research.

Proposed method

• Structured taxonomy across three dimensions: approach (image, text, paired data), data post-processing, and application.
• Summarize how large models (LLMs and diffusion models) enable augmentation.
• Discuss image, text, and multimodal prompt-driven and subject-driven augmentation methods.
• Evaluate successes and limitations across scenarios and propose future directions.
• Provide open-source resources and benchmarks related to large-model data augmentation.

Experimental results

Research questions

• RQ1 What are the main large-model-driven data augmentation methods across image, text, and paired data?
• RQ2 How do LLMs and diffusion models contribute to data augmentation in NLP, CV, and audio?
• RQ3 What data post-processing techniques accompany large-model augmentation, and what are their effects?
• RQ4 What are the current applications, challenges, and future directions for large-model-based data augmentation?

Key findings

• The study offers a comprehensive taxonomy of large-model-based data augmentation spanning approach, post-processing, and application.
• Large models enable richer, more diverse augmentation than traditional methods by leveraging semantic understanding and generative capabilities.
• The survey identifies successes and limitations of large-model augmentation across NLP, CV, and audio tasks.
• It discusses protocols, benchmarks, and quality metrics for evaluating augmented data and outlines grand challenges for the field.