[Paper Review] On the Diversity of Synthetic Data and its Impact on Training Large Language Models

Hao Chen, Abdul Waheed+5 more|arXiv (Cornell University)|Oct 19, 2024

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces LLM Cluster-agent, a novel framework that uses large language models to measure synthetic data diversity by clustering semantic characteristics, demonstrating that higher diversity in synthetic pre-training data significantly improves both pre-training and fine-tuning performance in LLMs, with effects on fine-tuning being more pronounced than on pre-training itself.

ABSTRACT

The rise of Large Language Models (LLMs) has accentuated the need for diverse, high-quality pre-training data. Synthetic data emerges as a viable solution to the challenges of data scarcity and inaccessibility. While previous literature has focused predominantly on the quality and quantity of real data, our work enables the measurement of diversity in synthetic data and explores its impact on LLM performance. We study the downstream effects of synthetic data diversity during both the pre-training and fine-tuning stages by introducing a new diversity metric, extit{LLM cluster-agent}, designed to evaluate the diversity of synthetic datasets. Through a series of controlled experiments with models of 350M and 1.4B parameters, we demonstrate that the proposed cluster-based LLM scoring of diversity correlates positively with both pre-training and supervised fine-tuning performance. Our findings also reveal that synthetic data diversity in pre-training affects supervised fine-tuning more significantly than pre-training itself, even for smaller models. We hope this study advances our understanding of the optimal use of synthetic data in LLM training and opens new avenues for efficient data generation processes.

Motivation & Objective

• To address the lack of reliable metrics for measuring diversity in synthetic text data used for LLM pre-training.
• To investigate how synthetic data diversity impacts the performance of large language models during pre-training and fine-tuning.
• To develop a scalable, LLM-based method for measuring diversity that outperforms traditional metrics on large-scale synthetic datasets.
• To determine whether diversity in synthetic data is a more critical factor than data quantity in determining LLM performance.

Proposed method

• Propose LLM Cluster-agent, a pipeline that uses LLMs to summarize and cluster semantic characteristics of randomly sampled text from a synthetic dataset.
• Design prompts that guide LLMs to identify salient features of data points and perform clustering based on these features, incorporating a self-verification mechanism to improve reliability.
• Compute an LLM cluster score from clustering results as a quantitative measure of dataset diversity.
• Validate the metric’s consistency and scalability across different LLMs and large-scale synthetic datasets where traditional metrics fail.
• Conduct controlled experiments using 620,000 Wikipedia topics to generate synthetic data with varying diversity levels, training 350M and 1.4B parameter models.
• Evaluate model performance on standard benchmarks after pre-training and supervised fine-tuning to correlate diversity scores with downstream results.

Figure 1: Linear regression of LLM cluster score and benchmark performance of (a) pre-trained 350M; (b) pre-trained 1.4B; (c) supervised fine-tuned 350M; and (d) supervised fine-tuned 1.4B models. Each scatter represents a synthetic dataset with size corresponding to the number of tokens.

Experimental results

Research questions

• RQ1 How does the diversity of synthetic pre-training data affect the performance of large language models during pre-training and fine-tuning?
• RQ2 Can an LLM-based method reliably measure the diversity of synthetic text data at scale, where traditional metrics fail?
• RQ3 Is synthetic data diversity more impactful on fine-tuning performance than on pre-training performance?
• RQ4 To what extent does synthetic data diversity outperform data quantity in determining LLM performance?

Key findings

• The LLM Cluster-agent diversity metric correlates positively with both pre-training and fine-tuning performance across models of 350M and 1.4B parameters.
• Synthetic data diversity has a stronger impact on supervised fine-tuning performance than on pre-training performance, even for smaller models.
• The proposed LLM-based diversity measurement method is consistent, scalable, and effective on large-scale synthetic data, outperforming traditional metrics that produce inconsistent results.
• The study reveals that diversity is a critical quality factor in synthetic data, with performance gains from high diversity outweighing gains from increased data quantity.
• Controlled experiments with 620,000 Wikipedia topics show that models pre-trained on diverse synthetic data generalize better during fine-tuning, confirming the metric’s predictive power.

Figure 2: Pipeline, prompt, and example outputs of the proposed LLM Cluster-agent. LLM Cluster-agent first generates metadata and metrics with attributes and scores that captures the underlying distribution and then uses these criteria to perform clustering with an extra self-verification step.