[Paper Review] FairFaceGAN: Fairness-aware Facial Image-to-Image Translation

Sunhee Hwang, Sungho Park+3 more|arXiv (Cornell University)|Dec 1, 2020

Generative Adversarial Networks and Image Synthesis41 references20 citations

TL;DR

FairFaceGAN is a fairness-aware GAN-based facial image-to-image translation model that decouples target attribute editing from protected attributes (e.g., gender, race, age) using two separate latents, ensuring minimal unwanted changes to protected attributes. It achieves state-of-the-art fairness with a 23.4% improvement in Equalized Odds and the lowest Fréchet Protected Attribute Distance (FPAD), while maintaining competitive image quality.

ABSTRACT

In this paper, we introduce FairFaceGAN, a fairness-aware facial Image-to-Image translation model, mitigating the problem of unwanted translation in protected attributes (e.g., gender, age, race) during facial attributes editing. Unlike existing models, FairFaceGAN learns fair representations with two separate latents - one related to the target attributes to translate, and the other unrelated to them. This strategy enables FairFaceGAN to separate the information about protected attributes and that of target attributes. It also prevents unwanted translation in protected attributes while target attributes editing. To evaluate the degree of fairness, we perform two types of experiments on CelebA dataset. First, we compare the fairness-aware classification performances when augmenting data by existing image translation methods and FairFaceGAN respectively. Moreover, we propose a new fairness metric, namely Frechet Protected Attribute Distance (FPAD), which measures how well protected attributes are preserved. Experimental results demonstrate that FairFaceGAN shows consistent improvements in terms of fairness over the existing image translation models. Further, we also evaluate image translation performances, where FairFaceGAN shows competitive results, compared to those of existing methods.

Motivation & Objective

• To address fairness issues in facial image-to-image translation where protected attributes (e.g., gender, race, age) are unintentionally altered during editing.
• To develop a method that preserves protected attributes during attribute translation without requiring protected attribute annotations.
• To evaluate fairness using both standard fairness metrics and a novel metric, FPAD, to measure protected attribute preservation.
• To demonstrate that the model maintains high image quality while significantly improving fairness over existing GAN-based methods.

Proposed method

• FairFaceGAN uses two disentangled latent spaces: one for target attribute editing and another for preserving protected attribute information.
• It introduces a Fair Representation Loss (FRL) to disentangle target attribute-related features from protected attribute-related features.
• A Protected Attribute Distance Loss (PADL) is applied to minimize distribution shifts in protected attributes between input and generated images.
• The model leverages a pre-trained classification model to extract protected attribute features without requiring ground-truth labels for these attributes.
• It employs perceptual loss to maintain high visual fidelity in generated images despite fairness constraints.
• The framework is trained end-to-end using adversarial loss, cycle consistency, and the proposed fairness-aware losses.

Experimental results

Research questions

• RQ1 Can a GAN-based image translation model preserve protected attributes (e.g., gender, race, age) during target attribute editing without requiring protected attribute annotations?
• RQ2 How does the disentanglement of target and protected attribute representations affect fairness in facial attribute translation?
• RQ3 To what extent does FairFaceGAN outperform existing models in fairness metrics such as Equality of Opportunity and Equalized Odds?
• RQ4 How effective is the proposed Fréchet Protected Attribute Distance (FPAD) metric in measuring protected attribute preservation compared to standard metrics?
• RQ5 Can FairFaceGAN generate high-quality images that are both visually realistic and fair in attribute editing?

Key findings

• FairFaceGAN achieved the lowest Fréchet Protected Attribute Distance (FPAD) of 1.24 on CelebA, indicating superior preservation of protected attributes compared to prior models.
• In fairness evaluation, FairFaceGAN improved Equalized Odds by 23.4% over the baseline, demonstrating significant reduction in demographic disparity.
• The model achieved a 92.11% accuracy in target attribute classification on generated images, outperforming other models and approaching the original test set accuracy of 88.88%.
• User study results showed that FairFaceGAN was ranked highest in both image quality and fairness, with 73 participants selecting it as the best in both categories.
• The model achieved competitive FID (13.8) and the best KID (0.0012) among compared methods, indicating strong image generation quality.
• Despite a slight trade-off in classification accuracy, the model's fairness gains were substantial, validating its core objective of bias mitigation in image translation.