[Paper Review] Evolutionary latent space search for driving human portrait generation

Benjamín Machín, Sergio Nesmachnow+1 more|arXiv (Cornell University)|Apr 25, 2022

Face recognition and analysis4 citations

TL;DR

This paper proposes an evolutionary algorithm to search the latent space of StyleGAN2 for generating synthetic human portraits highly similar to a target face image. By leveraging FaceNet for similarity evaluation and optimizing latent vectors through evolutionary search, the method generates diverse, realistic faces that deceive face recognition systems, reducing FaceNet embedding distance by up to 27.9% compared to real same-person images.

ABSTRACT

This article presents an evolutionary approach for synthetic human portraits generation based on the latent space exploration of a generative adversarial network. The idea is to produce different human face images very similar to a given target portrait. The approach applies StyleGAN2 for portrait generation and FaceNet for face similarity evaluation. The evolutionary search is based on exploring the real-coded latent space of StyleGAN2. The main results over both synthetic and real images indicate that the proposed approach generates accurate and diverse solutions, which represent realistic human portraits. The proposed research can contribute to improving the security of face recognition systems.

Motivation & Objective

• To develop an efficient method for generating synthetic human portraits that are visually and semantically similar to a given target face.
• To explore the high-dimensional latent space of StyleGAN2 using evolutionary algorithms to find vectors producing desired facial attributes.
• To evaluate the method's ability to deceive face recognition systems by minimizing FaceNet embedding distances between generated and target faces.
• To assess the diversity and quality of generated portraits across multiple target images and conditions.
• To analyze the computational cost and scalability of the proposed approach.

Proposed method

• An evolutionary algorithm (EA) is used to search the real-coded latent space of a pre-trained StyleGAN2 generator to produce human face images.
• FaceNet is employed as a fitness function to evaluate similarity between generated images and the target portrait by computing L2 distances between deep feature embeddings.
• The EA employs selection, crossover, and mutation operators to evolve latent vectors toward minimizing the L2 distance to the target embedding.
• The method includes optimizations such as pre-computing target embeddings and using efficient evaluation pipelines to reduce computational overhead.
• The search is conducted over a high-dimensional latent space (e.g., 512 dimensions), with population-based exploration to ensure diversity.
• The approach leverages pre-trained models (StyleGAN2 and FaceNet) to avoid costly retraining, enabling fast generation with minimal computational cost.

Experimental results

Research questions

• RQ1 Can an evolutionary algorithm effectively explore the latent space of StyleGAN2 to generate synthetic human portraits that are visually and semantically similar to a target face?
• RQ2 To what extent can the generated portraits deceive a pre-trained face recognition model like FaceNet, as measured by embedding similarity?
• RQ3 How diverse are the generated portraits across multiple runs and target images, and does the method maintain robustness in solution variety?
• RQ4 What is the computational cost of the proposed method, and how does it scale with different target images and hardware configurations?
• RQ5 Can the method generate multiple distinct yet accurate portraits of the same target person, demonstrating both fidelity and diversity?

Key findings

• The proposed method successfully generated synthetic human portraits that are visually similar to target faces, with L2 distances between FaceNet embeddings reduced by up to 27.9% compared to a different real image of the same person.
• For all tested instances (G2, W2, W3), the generated images had smaller L2 distances to the target than the baseline (a different image of the same person), confirming the method's ability to deceive face recognition systems.
• The method achieved high solution diversity, with pairwise L2 distances between embeddings of ten generated solutions ranging from 0.314 to 0.865, indicating varied yet valid outputs.
• The average diversity across instances was 0.491 ± 0.060 (G2), 0.645 ± 0.099 (W2), and 0.544 ± 0.070 (W3), demonstrating consistent robustness in generating varied portraits.
• The method computed accurate results in under 15 minutes on Google Colab, showing low computational cost due to the use of pre-trained models and efficient evaluation pipelines.
• The results confirm that evolutionary search in the latent space of StyleGAN2 can effectively produce realistic, diverse, and target-similar human portraits for adversarial purposes.