[Paper Review] Real-Time Adaptive Image Compression

Oren Rippel, Lubomir Bourdev|arXiv (Cornell University)|May 16, 2017

Advanced Image Processing TechniquesComputer Science20 references114 citations

TL;DR

A machine learning–based autoencoder for lossy image compression that achieves smaller files than JPEG, JPEG 2000, WebP and BPG while running in real-time on GPU.

ABSTRACT

We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all quality levels. At the same time, our codec is designed to be lightweight and deployable: for example, it can encode or decode the Kodak dataset in around 10ms per image on GPU. Our architecture is an autoencoder featuring pyramidal analysis, an adaptive coding module, and regularization of the expected codelength. We also supplement our approach with adversarial training specialized towards use in a compression setting: this enables us to produce visually pleasing reconstructions for very low bitrates.

Motivation & Objective

• Develop a lightweight, real-time ML-based image compressor that surpasses traditional codecs in compression efficiency.
• Enable adaptive bitstream control to meet target bitrate through learned representations.
• Incorporate adversarial training to improve perceptual quality at very low bitrates.
• Leverage a pyramidal, multiscale feature extraction and adaptive coding pipeline to enhance efficiency.

Proposed method

• Pyramidal multiscale feature extraction to capture structure across scales.
• Interscale alignment to fuse information across scales.
• Quantization, bitplane decomposition, and adaptive arithmetic coding for lossless coding of the quantized representation.
• Adaptive codelength regularization to steer the expected code length toward a target bitrate.
• Multiscale adversarial training with a GAN discriminator to encourage visually pleasing reconstructions.

Experimental results

Research questions

• RQ1 Can a neural compression model outperform traditional codecs at multiple bitrates while maintaining real-time encoding/decoding speeds?
• RQ2 How can adaptive codelength regularization and bitplane arithmetic coding be integrated to reach target bitrates efficiently?
• RQ3 Does multiscale adversarial training improve perceptual quality at very low bitrates without sacrificing compression performance?

Key findings

• The proposed ML-based codec typically produces files 2.5x smaller than JPEG and JPEG 2000, 2x smaller than WebP, and 1.7x smaller than BPG on Kodak PhotoCD and RAiSE-1k datasets across quality levels.
• On a GTX 980 Ti GPU, encoding/decoding times are around 9 ms and 10 ms per image, respectively, indicating real-time feasibility.
• The approach outperforms traditional codecs and prior ML-based methods in average MS-SSIM/Datasets’ performance across tested rates.
• Adaptive codelength regularization enables controlling the expected bitcount to target bitrates while keeping flexible bottleneck capacity.
• Multiscale adversarial training yields visually pleasing reconstructions at low bitrates by matching the statistics of ground-truth images.