[Paper Review] Deep Image Compression via End-to-End Learning

Haojie Liu, Tong Chen+3 more|arXiv (Cornell University)|Jun 1, 2018

Advanced Image Processing Techniques22 citations

TL;DR

This paper proposes a deep learning-based image compression method using CNNs that outperforms BPG, WebP, JPEG2000, and JPEG in MS-SSIM at equal bit rates. By combining perceptual and adversarial losses with a novel easy-to-hard transfer learning strategy for rate-distortion optimization, the method achieves 7.81% and 19.1% BD-rate reduction on Kodak and ETH Zurich’s P/M datasets, respectively.

ABSTRACT

We present a lossy image compression method based on deep convolutional neural networks (CNNs), which outperforms the existing BPG, WebP, JPEG2000 and JPEG as measured via multi-scale structural similarity (MS-SSIM), at the same bit rate. Currently, most of the CNNs based approaches train the network using a L2 loss between the reconstructions and the ground-truths in the pixel domain, which leads to over-smoothing results and visual quality degradation especially at a very low bit rate. Therefore, we improve the subjective quality with the combination of a perception loss and an adversarial loss additionally. To achieve better rate-distortion optimization (RDO), we also introduce an easy-to-hard transfer learning when adding quantization error and rate constraint. Finally, we evaluate our method on public Kodak and the Test Dataset P/M released by the Computer Vision Lab of ETH Zurich, resulting in averaged 7.81% and 19.1% BD-rate reduction over BPG, respectively.

Motivation & Objective

• To address the visual quality degradation and over-smoothing common in CNN-based image compression methods that use only L2 pixel-wise reconstruction loss.
• To improve subjective image quality at low bit rates by incorporating perceptual and adversarial losses in the training objective.
• To enhance rate-distortion optimization (RDO) through a novel easy-to-hard transfer learning strategy that gradually introduces quantization and rate constraints.
• To achieve state-of-the-art performance in image compression measured by MS-SSIM and BD-rate reduction on standard benchmark datasets.

Proposed method

• The method employs a deep convolutional neural network (CNN) for end-to-end image compression, learning joint representation of image features and quantization.
• A perception loss is used to preserve high-level semantic and structural content, reducing over-smoothing and improving visual quality.
• An adversarial loss is introduced to enhance texture realism and perceptual fidelity by training a discriminator to distinguish real from reconstructed images.
• An easy-to-hard transfer learning schedule is applied, gradually increasing the strength of quantization and rate constraints during training to stabilize optimization.
• The network is trained using a composite loss combining L2 reconstruction, perception, and adversarial losses, with rate constraints modeled via a differentiable rate estimation.
• The framework enables joint optimization of distortion and rate, achieving better rate-distortion trade-offs than conventional methods.

Experimental results

Research questions

• RQ1 Can combining perceptual and adversarial losses improve subjective image quality in deep image compression compared to standard L2 loss?
• RQ2 How does the proposed easy-to-hard transfer learning strategy affect the convergence and performance of rate-distortion optimization in CNN-based compression?
• RQ3 To what extent does the proposed method reduce BD-rate compared to state-of-the-art codecs like BPG and WebP on standard benchmarks?
• RQ4 Does the method maintain superior MS-SSIM performance across diverse image content at low bit rates?

Key findings

• The proposed method achieves an average 7.81% BD-rate reduction over BPG on the public Kodak dataset, indicating superior rate-distortion efficiency.
• On the ETH Zurich P/M test dataset, the method achieves a 19.1% BD-rate reduction over BPG, demonstrating strong performance on diverse and challenging image sets.
• The inclusion of perception and adversarial losses significantly improves visual quality, especially at low bit rates, reducing over-smoothing and enhancing texture details.
• The easy-to-hard transfer learning strategy enables stable training and better convergence, contributing to improved rate-distortion performance.
• The method outperforms BPG, WebP, JPEG2000, and JPEG in terms of MS-SSIM at equal bit rates, confirming its state-of-the-art performance.
• The results confirm that end-to-end learning with perceptual and adversarial supervision leads to more natural-looking reconstructions compared to standard L2-based training.