[Paper Review] Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet

Gonzalo Martínez, Lauren Watson+4 more|arXiv (Cornell University)|Jun 8, 2023

Computational Physics and Python Applications9 citations

TL;DR

The paper analyzes a worst-case feedback loop where AI-generated content from prior generations trains the next diffusion models, showing potential degradation in fidelity and diversity across datasets.

ABSTRACT

The rapid adoption of generative Artificial Intelligence (AI) tools that can generate realistic images or text, such as DALL-E, MidJourney, or ChatGPT, have put the societal impacts of these technologies at the center of public debate. These tools are possible due to the massive amount of data (text and images) that is publicly available through the Internet. At the same time, these generative AI tools become content creators that are already contributing to the data that is available to train future models. Therefore, future versions of generative AI tools will be trained with a mix of human-created and AI-generated content, causing a potential feedback loop between generative AI and public data repositories. This interaction raises many questions: how will future versions of generative AI tools behave when trained on a mixture of real and AI generated data? Will they evolve and improve with the new data sets or on the contrary will they degrade? Will evolution introduce biases or reduce diversity in subsequent generations of generative AI tools? What are the societal implications of the possible degradation of these models? Can we mitigate the effects of this feedback loop? In this document, we explore the effect of this interaction and report some initial results using simple diffusion models trained with various image datasets. Our results show that the quality and diversity of the generated images can degrade over time suggesting that incorporating AI-created data can have undesired effects on future versions of generative models.

Motivation & Objective

• Motivate and formalize questions about how AI-generated data affects future generative models via the Internet data ecosystem.
• Explore potential degradation, bias amplification, and loss of diversity through a feedback loop between Generative AI and training data.
• Provide initial empirical insights using simple diffusion models across multiple image datasets under a worst-case interaction model.
• Lay groundwork for more complex interaction models and broader datasets.

Proposed method

• Review diffusion models and relevant evaluation metrics for image generation.
• Propose a worst-case interaction model where each model version is trained on data generated by the previous version.
• Apply diffusion models (two variants: diffusion implicit and classifier-free guided) to MNIST, Oxford Flowers, and Caltech-UCSD Birds datasets.
• Evaluate fidelity and diversity using both small-image (classifier-based) metrics and Inception-based metrics (FID, precision, density, recall, coverage).
• Experiment across nine generations to observe trends in image quality and diversity over time.

Experimental results

Research questions

• RQ1 What happens to the fidelity and diversity of generative models when trained on data produced by earlier versions of themselves?
• RQ2 Do AI-generated data lead to degradation, stabilization, or collapse in image quality across datasets of different complexity?
• RQ3 How do dataset complexity and guidance strategies influence the evolution of generated content in a feedback loop?

Key findings

• Degradation in fidelity and diversity can occur over time for certain datasets when AI-generated data dominate training.
• Guidance strength strongly affects evolution: high guidance preserves sharp digits but reduces diversity; very low guidance leads to degeneration and loss of recognizable structure.
• No-guidance scenarios on complex color image datasets (flowers, birds) lead to progressive degradation and eventual collapse, with metrics like FID increasing over generations.
• Some datasets exhibit an initial degradation followed by stabilization, indicating dataset-dependent dynamics in the feedback loop.