[Paper Review] Geometry-Contrastive Generative Adversarial Network for Facial Expression Synthesis

Fengchun Qiao, Naiming Yao+4 more|arXiv (Cornell University)|Feb 6, 2018

Face recognition and analysis10 references25 citations

TL;DR

This paper proposes GC-GAN, a geometry-contrastive GAN that generates identity-preserving facial expressions from facial landmark inputs by learning a geometry-aware semantic manifold via contrastive learning. The method enables smooth expression transitions through geometry interpolation and achieves robust expression transfer across diverse face shapes.

ABSTRACT

In this paper, we propose a geometry-contrastive generative adversarial network GC-GAN for generating facial expression images conditioned on geometry information. Specifically, given an input face and a target expression designated by a set of facial landmarks, an identity-preserving face can be generated guided by the target expression. In order to embed facial geometry onto a semantic manifold, we incorporate contrastive learning into conditional GANs. Experiment results demonstrate that the manifold is sensitive to the changes of facial geometry both globally and locally. Benefited from the semantic manifold, dynamic smooth transitions between different facial expressions are exhibited via geometry interpolation. Furthermore, our method can also be applied in facial expression transfer even there exist big differences in face shape between target faces and driving faces.

Motivation & Objective

• To address the challenge of generating realistic, identity-preserving facial expressions from sparse geometric cues such as facial landmarks.
• To improve facial expression synthesis by learning a disentangled, geometry-sensitive semantic manifold that captures both global and local facial shape variations.
• To enable dynamic, smooth transitions between expressions through interpolation in the geometry-embedded latent space.
• To enhance facial expression transfer robustness when source and target faces differ significantly in shape.

Proposed method

• Integrates contrastive learning into conditional GANs to embed facial geometry into a disentangled semantic manifold.
• Uses facial landmarks as conditional input to guide the generator in producing identity-preserving facial images with target expressions.
• Learns a contrastive representation by contrasting positive pairs of facial geometry (e.g., similar expressions) and negative pairs (e.g., different expressions) in the latent space.
• Employs a Siamese network architecture to compute contrastive loss based on feature similarity between geometry-encoded representations.
• Enables expression interpolation by linearly interpolating between geometry codes in the learned semantic manifold.
• Supports expression transfer even when driving and target faces have large shape discrepancies by leveraging the geometry-embedded manifold.

Experimental results

Research questions

• RQ1 Can contrastive learning effectively embed facial geometry into a disentangled semantic manifold for facial expression synthesis?
• RQ2 Does the learned geometry-aware manifold support smooth and dynamic transitions between facial expressions?
• RQ3 Can the model generalize to facial expression transfer across faces with significant shape differences?
• RQ4 How does the integration of geometry and contrastive learning improve expression generation quality and identity preservation?

Key findings

• The learned semantic manifold is sensitive to both global and local changes in facial geometry, enabling precise expression control.
• Dynamic, smooth transitions between facial expressions are achieved through interpolation in the geometry-embedded latent space.
• The model generates high-fidelity, identity-preserving facial expressions even when the target expression is far from the source in appearance space.
• The method demonstrates robustness in facial expression transfer across diverse face shapes, outperforming baseline methods in generalization.