[论文解读] A Generative 3D Facial Model by Adversarial Training.

Victoria Fernández Abrevaya, Adnane Boukhayma+2 more|arXiv (Cornell University)|Feb 10, 2019

Face recognition and analysis被引用 4

一句话总结

该论文提出了一种使用3D生成器与2D判别器通过几何映射层连接的对抗性训练3D面部生成模型，实现了身份与表情的更好解耦，同时生成更多样化、无伪影的面部。该方法在标准基准测试中，在解耦性和样本多样性方面均达到了最先进性能。

ABSTRACT

Data-driven generative 3D face models are used to compactly encode facial shape data into meaningful parametric representations. A desirable property of these models is their ability to effectively decouple natural sources of variation, in particular identity and expression. While factorized representations have been proposed for that purpose, they are still limited in the variability they can capture and may present modeling artifacts when applied to tasks such as expression transfer. In this work, we explore a new direction with Generative Adversarial Networks and show that they contribute to better face modeling performances, especially in decoupling natural factors, while also achieving more diverse samples. To train the model we introduce a novel architecture that combines a 3D generator with a 2D discriminator that leverages conventional CNNs, where the two components are bridged by a geometry mapping layer. We further present a training scheme, based on auxiliary classifiers, to explicitly disentangle identity and expression attributes. Through quantitative and qualitative results on standard face datasets, we illustrate the benefits of our model and demonstrate that it outperforms competing state of the art methods in terms of decoupling and diversity.

研究动机与目标

• 开发一种数据驱动的3D面部生成模型，有效解耦身份与表情变化。
• 解决现有因子化模型的局限性，如建模伪影和表情迁移中的可变性有限问题。
• 通过对抗性训练提升3D面部生成的样本多样性和真实感。
• 提出一种新颖的架构与训练方案，显式解耦身份与表情属性。

提出的方法

• 3D生成器从潜在码生成详细的3D面部网格，学习合成逼真的面部几何形态。
• 2D判别器采用传统CNN实现，评估从生成的3D面部渲染出的2D图像的真实性。
• 几何映射层通过将3D网格坐标投影到2D图像空间，连接3D生成器与2D判别器，提供判别反馈。
• 引入基于辅助分类器的训练方案，在优化过程中显式监督身份与表情属性的解耦。
• 对抗性训练框架联合优化真实性与解耦性，提升泛化能力与多样性。
• 模型在标准3D面部数据集上端到端训练，学习有意义且因子化的表征。

实验结果

研究问题

• RQ1 与先前的因子化方法相比，对抗性训练是否能提升3D面部生成模型中身份与表情的解耦性能？
• RQ2 将3D生成器与2D判别器结合，如何增强3D面部合成的真实感与多样性？
• RQ3 几何映射层在多大程度上提升了3D与2D空间之间训练的稳定性与特征对齐？
• RQ4 辅助分类器能否在无需这些属性显式监督的情况下，有效引导身份与表情的解耦？
• RQ5 所提出的架构是否在解耦质量与样本多样性两方面均优于现有最先进方法？

主要发现

• 与竞争的最先进方法相比，该模型在标准基准测试中实现了更优的身份与表情属性解耦。
• 该模型生成的3D面部样本更具多样性，伪影更少，尤其在表情迁移任务中表现更优。
• 定量评估证实了解耦性能的提升，衡量身份与表情解耦的指标显著提高。
• 使用CNN实现的2D判别器相比纯3D判别器，能实现更稳定、更有效的对抗性训练。
• 基于辅助分类器的训练方案成功实现了身份与表情的解耦，且无需成对标注。
• 定性结果表明，生成的3D面部逼真且保真度高，身份保持一致，表情在多样化样本中可控。