[论文解读] Geometry-Contrastive GAN for Facial Expression Transfer

Fengchun Qiao, Naiming Yao+4 more|arXiv (Cornell University)|Feb 6, 2018

Face recognition and analysis参考文献 24被引用 51

一句话总结

GC-GAN 通过将几何信息与对比学习嵌入，学习用于面部表情的连续语义潜在空间，从而在不同主体之间实现保留身份的表情传输。

ABSTRACT

In this paper, we propose a Geometry-Contrastive Generative Adversarial Network (GC-GAN) for transferring continuous emotions across different subjects. Given an input face with certain emotion and a target facial expression from another subject, GC-GAN can generate an identity-preserving face with the target expression. Geometry information is introduced into cGANs as continuous conditions to guide the generation of facial expressions. In order to handle the misalignment across different subjects or emotions, contrastive learning is used to transform geometry manifold into an embedded semantic manifold of facial expressions. Therefore, the embedded geometry is injected into the latent space of GANs and control the emotion generation effectively. Experimental results demonstrate that our proposed method can be applied in facial expression transfer even there exist big differences in facial shapes and expressions between different subjects.

研究动机与目标

• Motivate facial expression transfer across subjects with continuous emotion representations.
• Introduce geometry as a continuous condition to guide expression synthesis in GANs.
• Embed facial geometry into a semantic latent space using contrastive learning to reduce misalignment across subjects.
• Ensure Lipschitz continuity of the generator with respect to expression-conditioning latent variables.
• Demonstrate effectiveness on multiple datasets and through cross-dataset transfer scenarios.

提出的方法

• Introduce a three-component GC-GAN: a geometry embedding network E, an image generator G, and an image discriminator D.
• Embed geometry using contrastive learning to obtain z_g = E_enc(g_k^v), forming a semantic expression manifold.
• Concatenate identity latent z_i (from I_j^u) with geometry latent z_g to form the generator input, enforcing Lipschitz continuity.
• Train E with a contrastive loss L_contr to pull same-expression landmarks together and push different expressions apart.
• Use adversarial losses (WGAN-GP) plus reconstruction losses (L_ir for images, L_gr for landmarks) to train G and E.
• Proceed in two stages: pretrain E with L_contr and L_gr, then train G and D with fixed E, optimizing L_ir and L_adv.

实验结果

研究问题

• RQ1 Can continuous geometry embeddings guide expression transfer across subjects while preserving identity?
• RQ2 Does contrastive embedding of facial geometry yield a semantic, Lipschitz-continuous expression latent space that improves transfer quality?
• RQ3 How does GC-GAN perform across diverse datasets and in cross-dataset expression transfer scenarios?
• RQ4 What is the impact of the reconstruction and adversarial losses on synthesis quality and expression fidelity?

主要发现

• GC-GAN enables identity-preserving facial expression transfer guided by an embedded geometry latent space.
• Incorporating L_ir, L_gr, and L_contr improves SSIM and PSNR over ablations and a baselines like CDAAE across Multi-PIE, CK+, and BU-4DFE.
• The geometry embedding yields a Lipschitz-continuous mapping with respect to expression conditions, facilitating smooth expression transitions.
• Cross-dataset experiments (CelebA guiding CelebA-derived landmarks) show GC-GAN can transfer expressions while preserving identity.
• Ablations show removing any of the three losses degrades performance, highlighting the contribution of reconstruction and contrastive guidance.