[论文解读] A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot

Milad Abdollahzadeh, Guimeng Liu+4 more|arXiv (Cornell University)|Jul 26, 2023

Machine Learning in Healthcare被引用 10

一句话总结

对在数据约束下的生成建模（GM-DC）的综合综述，详细介绍在GANs、VAEs和扩散模型方面的任务、数据约束、方法、挑战及未来方向。

ABSTRACT

Generative modeling in machine learning aims to synthesize new data samples that are statistically similar to those observed during training. While conventional generative models such as GANs and diffusion models typically assume access to large and diverse datasets, many real-world applications (e.g. in medicine, satellite imaging, and artistic domains) operate under limited data availability and strict constraints. In this survey, we examine Generative Modeling under Data Constraint (GM-DC), which includes limited-data, few-shot, and zero-shot settings. We present a unified perspective on the key challenges in GM-DC, including overfitting, frequency bias, and incompatible knowledge transfer, and discuss how these issues impact model performance. To systematically analyze this growing field, we introduce two novel taxonomies: one categorizing GM-DC tasks (e.g. unconditional vs. conditional generation, cross-domain adaptation, and subject-driven modeling), and another organizing methodological approaches (e.g. transfer learning, data augmentation, meta-learning, and frequency-aware modeling). Our study reviews over 230 papers, offering a comprehensive view across generative model types and constraint scenarios. We further analyze task-approach-method interactions using a Sankey diagram and highlight promising directions for future work, including adaptation of foundation models, holistic evaluation frameworks, and data-centric strategies for sample selection. This survey provides a timely and practical roadmap for researchers and practitioners aiming to advance generative modeling under limited data. Project website: https://sutd-visual-computing-group.github.io/gmdc-survey/.

研究动机与目标

• GM-DC 的研究背景与动机，其中数据获取具有挑战性（如医疗保健领域）。
• 引入两种分类体系：GM-DC 任务和 GM-DC 方法，并分析它们之间的相互作用。
• 突出 GM-DC 研究中的趋势、空白与未来方向。
• 提供对 GM-DC 工作的有序总结以及带有交互式全景可视化的项目网站。

提出的方法

• 对跨越 GANs、VAEs 和扩散模型的 GM-DC 文献进行评估与综合。
• 提出 GM-DC 的任务分类（uGM-1 到 cGM-3、IGM、SGM）及相应的数据约束映射。
• 提出方法分类（迁移学习、数据增强、网络架构、多任务目标、频率利用、元学习等）。
• 分析 GM-DC 任务与方法之间的相互作用，辅以 Sankey 图和图表等可视化。

Figure 1 . Research Landscape of GM-DC. The figure shows the interaction between GM-DC tasks and approaches (main and sub categories), and GM-DC methods. Tasks are defined in our proposed taxonomy in Tab. 2 , and approaches in our proposed taxonomy in Tab. LABEL:tab:approaches . An interactive versi

实验结果

研究问题

• RQ1 研究了哪些 GM-DC 任务，以及它们在无条件/有条件和跨领域设置中的定义方式？
• RQ2 哪些数据约束模态（LD、FS、ZS）较为普遍，它们如何影响方法选择？
• RQ3 在数据约束下，哪些 GM-DC 方法在知识迁移、数据增广、架构设计和元学习方面最有效？
• RQ4 GM-DC 的关键挑战与待解决的问题有哪些？哪些方向在未来工作中最具潜力？

主要发现

• GM-DC 研究跨越多种生成模型家族（GANs、VAEs、Diffusion Models），覆盖多种数据约束设置。
• 两类详细的分类体系（GM-DC 任务和 GM-DC 方法）对文献进行了组织，并揭示任务与方法之间的互动。
• 迁移学习、数据增强和文本/语言引导的适应性是数据有限情况下 GM-DC 的核心策略。
• 元学习与多任务目标在人机未见领域或类别的适应中显示出有效性。
• 该综述强调领域近似、跨域适应和知识迁移是塑造 GM-DC 结果的关键因素。

Figure 2 . Overall publications statistics in GM-DC. GM-DC Publications (Left): GM-DC publication trends indicate rising interest in this area. We remark that the previous survey (Li et al . , 2022c ) only covers $\sim$ 27% of publications discussed in our survey. Publication Venues (Right): The dis