[Paper Review] High-Quality Face Image SR Using Conditional Generative Adversarial Networks

Bin Huang, Weihai Chen+2 more|arXiv (Cornell University)|Jul 4, 2017

Advanced Image Processing Techniques24 references22 citations

TL;DR

This paper proposes Face Conditional Generative Adversarial Network (FCGAN), an end-to-end framework that generates high-resolution (4×) face images from low-resolution inputs using boundary equilibrium GANs. By conditioning the generator on the LR image instead of random noise and employing L1 loss and skip connections, FCGAN achieves state-of-the-art PSNR (32.42) and superior visual quality without facial priors or pre-processing.

ABSTRACT

We propose a novel single face image super-resolution method, which named Face Conditional Generative Adversarial Network(FCGAN), based on boundary equilibrium generative adversarial networks. Without taking any facial prior information, our method can generate a high-resolution face image from a low-resolution one. Compared with existing studies, both our training and testing phases are end-to-end pipeline with little pre/post-processing. To enhance the convergence speed and strengthen feature propagation, skip-layer connection is further employed in the generative and discriminative networks. Extensive experiments demonstrate that our model achieves competitive performance compared with state-of-the-art models.

Motivation & Objective

• To develop an end-to-end super-resolution method for single face images that does not rely on facial priors such as landmarks or alignment.
• To adapt the Boundary Equilibrium GAN (BEGAN) framework for face image super-resolution by conditioning generation on low-resolution input images instead of random noise.
• To improve training convergence and feature propagation through skip-layer connections in both generator and discriminator networks.
• To achieve high visual quality and competitive quantitative performance (PSNR) on the CelebA dataset without pre/post-processing steps.

Proposed method

• The generator network takes a low-resolution face image ($32\times32$) as input and generates a high-resolution output ($128\times128$) using a U-Net-like architecture with skip connections.
• The discriminator network is trained to distinguish between real high-resolution images and generated high-resolution images conditioned on the same low-resolution input.
• The model uses a conditional GAN loss with pixel-wise $L_1$ reconstruction loss to stabilize training and improve perceptual quality.
• The training objective combines the standard GAN loss with a gradient penalty term to ensure equilibrium between generator and discriminator, following the BEGAN framework.
• Skip connections are applied in both generator and discriminator to enhance feature propagation and accelerate convergence.
• The framework is trained end-to-end using the CelebA dataset, with input-output pairs created via bicubic downsampling and interpolation.

Experimental results

Research questions

• RQ1 Can a conditional GAN framework based on BEGAN generate high-quality face images from low-resolution inputs without relying on facial priors?
• RQ2 How does conditioning the generator on the LR image instead of random noise affect the quality and fidelity of generated HR face images?
• RQ3 To what extent do skip connections and $L_1$ loss improve convergence speed and visual quality in face super-resolution?
• RQ4 How does FCGAN compare to state-of-the-art methods in terms of both quantitative metrics (PSNR) and qualitative image quality?
• RQ5 Can the model generalize robustly across variations in pose, expression, lighting, and occlusion without alignment or preprocessing?

Key findings

• FCGAN achieves a PSNR of 32.42 on the CelebA dataset, outperforming state-of-the-art methods including LapSRN (32.13) and FSRCNN (31.92).
• Visual comparisons show that FCGAN generates sharper facial details and more realistic textures than bicubic interpolation, pix2pix, and other SOTA models.
• The model generates high-quality HR face images consistently across diverse conditions, including varying poses, expressions, lighting, and occlusions (e.g., glasses or hats).
• The use of skip connections significantly improves training convergence speed and stability compared to deeper CNNs without residual connections.
• The absence of pre-processing steps like face alignment or landmark detection does not degrade performance, demonstrating the robustness of the conditional GAN approach.
• The model is fully end-to-end, requiring no external priors or post-processing, and generates HR images directly from LR inputs in a single forward pass.