[论文解读] Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art

Tanujit Chakraborty, Ujjwal Reddy K S+3 more|arXiv (Cornell University)|Aug 30, 2023

Generative Adversarial Networks and Image Synthesis被引用 8

一句话总结

一个对GANs自2014年 inception 至今的综合综述， covering 架构、理论、评估、训练挑战、跨领域应用，以及与新兴DL模型的混合化。

ABSTRACT

Since their inception in 2014, Generative Adversarial Networks (GANs) have rapidly emerged as powerful tools for generating realistic and diverse data across various domains, including computer vision and other applied areas. Consisting of a discriminative network and a generative network engaged in a Minimax game, GANs have revolutionized the field of generative modeling. In February 2018, GAN secured the leading spot on the ``Top Ten Global Breakthrough Technologies List'' issued by the Massachusetts Science and Technology Review. Over the years, numerous advancements have been proposed, leading to a rich array of GAN variants, such as conditional GAN, Wasserstein GAN, CycleGAN, and StyleGAN, among many others. This survey aims to provide a general overview of GANs, summarizing the latent architecture, validation metrics, and application areas of the most widely recognized variants. We also delve into recent theoretical developments, exploring the profound connection between the adversarial principle underlying GAN and Jensen-Shannon divergence, while discussing the optimality characteristics of the GAN framework. The efficiency of GAN variants and their model architectures will be evaluated along with training obstacles as well as training solutions. In addition, a detailed discussion will be provided, examining the integration of GANs with newly developed deep learning frameworks such as Transformers, Physics-Informed Neural Networks, Large Language models, and Diffusion models. Finally, we reveal several issues as well as future research outlines in this field.

研究动机与目标

• 在过去十年中对GAN架构及其演变提供广泛概述。
• 总结将对抗性训练与散度度量及最优性相关的关键理论发展。
• 评估指标与实际训练挑战的综述，包括稳定性和模式崩溃。
• 讨论跨领域的应用（计算机视觉、NLP、时间序列、医学、城市规划、地球科学）以及与新框架的实际集成。
• 概述未来研究方向以及与 Transformers、PINNs、LLMs、扩散模型的潜在混合化。

提出的方法

• 对经典GAN工作及其变体（条件GAN、Wasserstein GAN、CycleGAN、StyleGAN等）进行系统文献综述。
• 采用按时间顺序的组织，以展示十年来的架构与方法学进展。
• 对对抗目标、Jensen-Shannon 散度及相关最优性考虑进行理论讨论。
• 在领域特异性能考虑下进行评估与局限性评估。
• 讨论训练挑战及提出的补救措施，包括稳定性改进和替代损失函数。
• 分析与新DL范式（Transformers、PINNs、LLMs、Diffusion模型）的整合及其对GAN有效性的影响。

Figure 1: Architecture of GANs and its primary functions. In this example, different analytical tasks of GANs are categorized into synthetic data generation, style transfer, data augmentation, and anomaly detection.

实验结果

研究问题

• RQ1 在过去十年中发展出来的主要GAN变体有哪些？它们解决了哪些问题？
• RQ2 支撑GANs的关键理论见解有哪些，包括与散度度量和最优性的联系？
• RQ3 在不同领域评估GAN生成数据所使用的指标与评估策略有哪些？
• RQ4 哪些训练挑战限制GAN性能，已有的解决方案有哪些？
• RQ5 如何将GAN与新兴深度学习框架集成，以推动新应用中的合成数据生成？

主要发现

• GAN已经从基础结构演进为专门变体（如条件、Wasserstein、CycleGAN、StyleGAN）以解决质量、多样性和条件化需求。
• 训练不稳定性和模式崩溃仍是核心挑战，提出了损失函数与结构改进以提升稳定性。
• 生成数据的偏差与伦理问题被认识到需要仔细评估与缓解。
• 与 Transformers、PINNs、LLMs、扩散模型的混合方法在扩展GAN能力与应用方面显示出潜力。
• GAN在多领域得到应用，包括计算机视觉、NLP、时间序列、医学、地球科学、城市规划等，用于生成、扩增、风格迁移和仿真。

Figure 2: Timeline of the application-based GAN architectures reviewed in this study