[Paper Review] High-Fidelity Generative Image Compression

Fabian Mentzer, George Toderici+2 more|arXiv (Cornell University)|Jun 17, 2020

Advanced Data Compression TechniquesComputer Science49 references214 citations

TL;DR

The paper introduces HiFiC, a GAN-based learned image compression system that delivers perceptually high-fidelity reconstructions at high resolutions and across a range of bitrates, outperforming traditional codecs like BPG in user studies and with diverse perceptual metrics.

ABSTRACT

We extensively study how to combine Generative Adversarial Networks and learned compression to obtain a state-of-the-art generative lossy compression system. In particular, we investigate normalization layers, generator and discriminator architectures, training strategies, as well as perceptual losses. In contrast to previous work, i) we obtain visually pleasing reconstructions that are perceptually similar to the input, ii) we operate in a broad range of bitrates, and iii) our approach can be applied to high-resolution images. We bridge the gap between rate-distortion-perception theory and practice by evaluating our approach both quantitatively with various perceptual metrics, and with a user study. The study shows that our method is preferred to previous approaches even if they use more than 2x the bitrate.

Motivation & Objective

• Motivate perceptually faithful lossy compression at high resolutions beyond traditional rate-distortion optimization.
• Develop a GAN-based compression framework that aligns distributional Reconstruction with input images.
• Evaluate the approach across multiple perceptual metrics and user studies to bridge rate-distortion-perception theory to practice.
• Assess architectural choices (normalization, generator/discriminator design) and loss terms that affect perceptual quality and stability.

Proposed method

• Formulate end-to-end learned compression augmented with a conditional GAN to guide perceptual quality.
• Use a distortion loss combining MSE and LPIPS-based perceptual distortion with a learned rate via a probability model and entropy coding.
• Incorporate a rate-target mechanism to control average bitrate through an adaptive lambda prime during training.
• Adopt a ChannelNorm normalization to stabilize training and replace InstanceNorm to improve cross-resolution consistency.
• Condition the discriminator on the latent representation to form a conditional GAN, and use a hyperprior-based probability model for latent entropy modeling.
• Train with a two-stage initialization and ablation studies to analyze the effects of GAN, perceptual loss, and generator capacity.

Experimental results

Research questions

• RQ1 How can a GAN-based compression framework achieve reconstructions that are perceptually faithful to high-resolution inputs?
• RQ2 What architectural and loss-function choices best balance rate, distortion, and perceptual quality in neural image compression?
• RQ3 Do perceptual metrics and human evaluations align in ranking GAN-based compression methods across bitrates and datasets?
• RQ4 How does conditioning the discriminator and using advanced normalization affect training stability and perceptual outcomes?

Key findings

• HiFiC yields higher perceptual fidelity than baselines at similar or even lower bitrates, with user studies preferring HiFiC reconstructions to BPG at 2x the bitrate.
• Across metrics like FID, KID, NIQE, LPIPS, and PSNR/MS-SSIM, the GAN-based approach improves perceptual indices while sacrificing traditional distortion metrics, consistent with rate-distortion-perception theory.
• A conditional discriminator and ChannelNorm stabilize training and improve sharpness and consistency across resolutions.
• Varying the perceptual distortion component (LPIPS-based) and GAN weight shows a trade-off where higher perceptual quality increases distortion, especially at lower bitrates; this trade-off is navigable via the beta parameter.
• The approach scales to high-resolution images (up to 2000×2000) and is validated through quantitative metrics and a user preference study on diverse datasets.