[论文解读] Training Deep Face Recognition Systems with Synthetic Data

Adam Kortylewski, Schneider, Andreas+4 more|arXiv (Cornell University)|Feb 16, 2018

Face recognition and analysis参考文献 31被引用 51

一句话总结

这篇论文表明来自3D Morphable Face Model的合成数据可以加强深度人脸识别，减少对真实数据的需求，并且在对真实数据进行微调后，能够缩小合成训练与真实训练模型之间的差距。

ABSTRACT

Recent advances in deep learning have significantly increased the performance of face recognition systems. The performance and reliability of these models depend heavily on the amount and quality of the training data. However, the collection of annotated large datasets does not scale well and the control over the quality of the data decreases with the size of the dataset. In this work, we explore how synthetically generated data can be used to decrease the number of real-world images needed for training deep face recognition systems. In particular, we make use of a 3D morphable face model for the generation of images with arbitrary amounts of facial identities and with full control over image variations, such as pose, illumination, and background. In our experiments with an off-the-shelf face recognition software we observe the following phenomena: 1) The amount of real training data needed to train competitive deep face recognition systems can be reduced significantly. 2) Combining large-scale real-world data with synthetic data leads to an increased performance. 3) Models trained only on synthetic data with strong variations in pose, illumination, and background perform very well across different datasets even without dataset adaptation. 4) The real-to-virtual performance gap can be closed when using synthetic data for pre-training, followed by fine-tuning with real-world images. 5) There are no observable negative effects of pre-training with synthetic data. Thus, any face recognition system in our experiments benefits from using synthetic face images. The synthetic data generator, as well as all experiments, are publicly available.

研究动机与目标

• 激发并量化合成生成的面部图像是否能够支持并改进深度人脸识别系统。
• 刻画基于合成数据训练的模型与基于真实数据训练的模型在真实场景中的性能差距。
• 演示合成数据如何降低对真实数据的需求或通过增广真实数据来提升基准性能。
• 研究合成数据属性的变化（姿态分布、身份数量）如何影响现实世界的性能。
• 提供可复现的合成数据生成器并分析合成到真实转移的局限性。

提出的方法

• 通过对统计性的3D Morphable Model（Basel Face Model）在形状、颜色、姿态、照明和表情上进行采样来生成大规模的合成人脸图像。
• 使用光照先验和随机背景进行渲染，以产生现实世界的变异性。
• 在合成数据（SYN-1M）上使用 OpenFace 框架训练 FaceNet-NN4，且不进行真实数据增强或适应。
• 在 CMU-Multipie、LFW 和 IJB-A 上使用 128-d 嵌入的余弦相似度来衡量识别性能。
• 在不同量级的真实数据上对合成预训练模型进行微调（Casia 子集），以评估差距缩小和性能提升。

实验结果

研究问题

• RQ1 Can synthetic data alone achieve competitive face recognition performance on standard benchmarks?
• RQ2 What is the real-to-virtual gap between models trained on synthetic data versus real data across benchmarks like CMU-Multipie, LFW, and IJB-A?
• RQ3 Can pre-training on synthetic data plus fine-tuning on real data close or bridge the gap to real-data-only models?
• RQ4 How do synthetic data characteristics (pose distribution, number of identities) influence real-world performance?
• RQ5 What is the optimal balance between synthetic pre-training and real-data fine-tuning for best transfer to real-world datasets?

主要发现

• A significant real-to-virtual performance gap exists when training on synthetic data alone, especially on LFW and IJB-A.
• Pre-training on synthetic data followed by fine-tuning with real data closes the gap and can outperform real-data-only models on multiple benchmarks.
• Using synthetic data allows reducing the amount of real data needed to reach competitive performance; e.g., on LFW the gap is reduced with about 100K real images for fine-tuning.
• Combining synthetic data with real data yields performance gains across Multipie, LFW, and IJB-A, often exceeding real data-only baselines.
• The synthetic data generator, based on 200 real 3D scans, can generate diverse identities and variations; pose and background diversity are crucial for transfer gains.
• Increasing the number of identities and maintaining broad pose variation in synthetic data further improves transfer to real datasets.