[论文解读] FreGAN: Exploiting Frequency Components for Training GANs under Limited Data

Mengping Yang, Zhe Wang+2 more|arXiv (Cornell University)|Oct 11, 2022

Advanced Neural Network Applications被引用 27

一句话总结

FreGAN 利用频率分量在数据有限的情况下训练 GAN，在与若干基线相比的多个风格下实现有竞争力的 LPIPS 分数。

ABSTRACT

Training GANs under limited data often leads to discriminator overfitting and memorization issues, causing divergent training. Existing approaches mitigate the overfitting by employing data augmentations, model regularization, or attention mechanisms. However, they ignore the frequency bias of GANs and take poor consideration towards frequency information, especially high-frequency signals that contain rich details. To fully utilize the frequency information of limited data, this paper proposes FreGAN, which raises the model's frequency awareness and draws more attention to producing high-frequency signals, facilitating high-quality generation. In addition to exploiting both real and generated images' frequency information, we also involve the frequency signals of real images as a self-supervised constraint, which alleviates the GAN disequilibrium and encourages the generator to synthesize adequate rather than arbitrary frequency signals. Extensive results demonstrate the superiority and effectiveness of our FreGAN in ameliorating generation quality in the low-data regime (especially when training data is less than 100). Besides, FreGAN can be seamlessly applied to existing regularization and attention mechanism models to further boost the performance.

研究动机与目标

• 解决在数据有限的情况下训练高质量 GAN 的挑战。
• 利用频域信息来提升生成质量。
• 在多样化风格下使用 LPIPS 将 FreGAN 与强基线进行比较。
• 提供小样本数据集上鲁棒性的定量证据。

提出的方法

• 提出 FreGAN，一个为数据受限情形设计的频率感知 GAN 训练框架。
• 使用 LPIPS 作为主要指标，在五种风格上评估性能。
• 与 StyleGAN2、ADA、APA、DiffAug 和 FastGAN 进行对比。
• 给出定量结果，展示 FreGAN 相对于基线的性能表现。

实验结果

研究问题

• RQ1 在数据有限的条件下，FreGAN 是否在感知质量（LPIPS）方面优于强基线？
• RQ2 FreGAN 在多样化视觉风格（AnimeFace、ArtPainting、Moongate、Flat、Fauvism）上表现如何？
• RQ3 基于频率的训练相对于传统的数据增强或正则化对 LPIPS 的影响有多大？
• RQ4 在不同数据 regime 和风格数据集上，FreGAN 的性能是否具有一致性？

主要发现

方法	AnimeFace LPIPS	ArtPainting LPIPS	Moongate LPIPS	Flat LPIPS	Fauvism LPIPS
StyleGAN2	0.4253 ± 0.0020	0.7244 ± 0.0009	0.7047 ± 0.0026	0.6223 ± 0.0026	0.6344 ± 0.0009
ADA	0.5611 ± 0.0015	0.8102 ± 0.0015	0.6418 ± 0.0015	0.7288 ± 0.0017	0.6509 ± 0.0014
APA	0.5491 ± 0.0017	0.8062 ± 0.0014	0.7235 ± 0.0016	0.6317 ± 0.0022	0.6848 ± 0.0014
DiffAug	0.4926 ± 0.0005	0.7717 ± 0.0016	0.5880 ± 0.0015	0.4403 ± 0.0005	0.6117 ± 0.0023
FastGAN	0.6188 ± 0.0011	0.8344 ± 0.0015	0.6603 ± 0.0010	0.7939 ± 0.0016	0.7028 ± 0.0010
FreGAN (Ours)	0.6191 ± 0.0010	0.8439 ± 0.0016	0.6673 ± 0.0016	0.7952 ± 0.0011	0.7028 ± 0.0010

• FreGAN 在五种风格上实现了 0.6191、0.8439、0.6673、0.7952、0.7028 的 LPIPS。
• FreGAN 在多个风格上与若干基线（StyleGAN2、ADA、APA、DiffAug、FastGAN）具有竞争力或超越。
• 对于 FreGAN，LPIPS 值在不同风格下接近或优于若干基线，具体取决于风格。