[Paper Review] Lossy Image Compression with Compressive Autoencoders

Lucas Theis, Wenzhe Shi+2 more|arXiv (Cornell University)|Mar 1, 2017

Advanced Data Compression Techniques30 references259 citations

TL;DR

The paper introduces compressive autoencoders (CAEs) for end-to-end lossy image compression, addressing non-differentiability in quantization with gradient-friendly surrogates and entropy modeling, and showing competitive results to JPEG 2000 with efficient high-resolution decoding.

ABSTRACT

We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to address this need, but are difficult to optimize directly due to the inherent non-differentiabilty of the compression loss. We here show that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs. Our network is furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images. This is in contrast to previous work on autoencoders for compression using coarser approximations, shallower architectures, computationally expensive methods, or focusing on small images.

Motivation & Objective

• Motivate the need for flexible lossy compression beyond traditional codecs.
• Propose a learnable framework (CAE) to optimize rate-distortion jointly.
• Develop differentiable strategies to handle quantization and entropy coding.
• Demonstrate competitive performance on standard datasets and analyze perceptual quality.

Proposed method

• Define a compressive autoencoder with encoder f, decoder g, and entropy model Q.
• Optimize the rate-distortion objective -log2 Q([f(x)]) + beta * d(x, g([f(x)])) with a differentiable approximation.
• Replace the non-differentiable rounding-based quantization gradient with a simple differentiable surrogate for backpropagation.
• Upper-bound the non-differentiable bit cost via a continuous density q and Jensen’s inequality to enable gradient-based training.
• Model the distribution of encoded coefficients with Gaussian scale mixtures for entropy coding.
• Use a sub-pixel convolutional architecture and incremental/finetuning training to achieve efficient high-resolution performance.
• Provide flexible bitrate control by learning scale parameters that adjust the bitrate across rate-distortion tradeoffs.

Experimental results

Research questions

• RQ1 Can CAEs reach competitive rate-distortion performance compared to JPEG 2000 and RNN-based methods for natural images?
• RQ2 How can non-differentiable quantization and entropy coding be effectively handled during end-to-end training?
• RQ3 Does an efficient architecture (sub-pixel upsampling) enable near real-time decoding of high-resolution images?
• RQ4 Does incremental training and fine-tuning across rate-distortion settings improve stability and performance?
• RQ5 How do CAEs perform on perceptual quality metrics (SSIM, MS-SSIM, MOS) compared to standard codecs?

Key findings

• CAE achieves performance similar to or better than JPEG 2000 on perceptual metrics like SSIM and MOS on Kodak images.
• The CAE outperforms JPEG 2000 in SSIM and MOS at certain bitrates and provides smoother artefacts than JPEG 2000.
• An efficient convolutional architecture with sub-pixel upsampling enables high-resolution decoding on consumer hardware.
• Incremental training and learned scale parameters give flexible, fine-grained control over bitrate without training a large number of separate models.
• End-to-end optimization allows the CAE to adapt to content-specific tasks and metrics beyond traditional codecs.