[Paper Review] Learning Accurate Entropy Model with Global Reference for Image Compression

Yichen Qian, Zhiyu Tan+6 more|arXiv (Cornell University)|Oct 16, 2020

Advanced Image Processing Techniques22 references35 citations

TL;DR

Introduces a global reference-based entropy model and a mean-shifting GDN (GSDN) to improve rate-distortion performance in learned image compression. Demonstrates that combining local context, global references, and hyperprior yields state-of-the-art results on standard benchmarks.

ABSTRACT

In recent deep image compression neural networks, the entropy model plays a critical role in estimating the prior distribution of deep image encodings. Existing methods combine hyperprior with local context in the entropy estimation function. This greatly limits their performance due to the absence of a global vision. In this work, we propose a novel Global Reference Model for image compression to effectively leverage both the local and the global context information, leading to an enhanced compression rate. The proposed method scans decoded latents and then finds the most relevant latent to assist the distribution estimating of the current latent. A by-product of this work is the innovation of a mean-shifting GDN module that further improves the performance. Experimental results demonstrate that the proposed model outperforms the rate-distortion performance of most of the state-of-the-art methods in the industry.

Motivation & Objective

• Motivate and address non-local spatial redundancy in latent representations for learned image compression.
• Develop a global reference module to augment local context and hyperprior in entropy modeling.
• Introduce GSDN to correct mean-shifting in GDN and improve latent distribution alignment.
• Demonstrate that the combined model yields RD gains over state-of-the-art codecs and learning-based methods.

Proposed method

• Extend the autoencoder-based image compression framework with a combined entropy model that fuses local context, global reference, and hyperprior.
• Model each latent as Gaussian with mean and scale parameters conditioned on previous features, plus a hyperprior term (p_hat_y) as in prior work.
• Implement a reference-based module that searches decoded latents (masked patches) to find relevant latents for each target latent using cosine similarity and a confidence map U to weight the reference.
• Use a generalized subtractive and divisive normalization (GSDN) to replace GDN, correcting a mean-shifting issue in latent distributions and enabling invertibility in decoding.
• Ensemble local, global, and hyperprior features stage-by-stage to progressively refine Gaussian parameters (mu, sigma).
• Train with rate-distortion objective, exploring both MSE and MS-SSIM distortions, and compare against BPG/JPEG and prior learned methods.

Experimental results

Research questions

• RQ1 Can a global reference mechanism improve entropy estimation by exploiting non-local spatial redundancy in latent representations?
• RQ2 Does integrating a global reference with local context and hyperprior outperform context-only and hyperprior-based entropy models in learned image compression?
• RQ3 What is the impact of replacing GDN with GSDN on the distribution of latents and overall compression performance?
• RQ4 How does the proposed reference module behave under different distortion metrics (MSE vs MS-SSIM) and bitrates?
• RQ5 Is the combined model scalable in terms of complexity while providing RD gains over state-of-the-art methods?

Key findings

• The full model outperforms state-of-the-art codecs and learning-based methods on PSNR and MS-SSIM RD performance on Kodak and other benchmarks.
• Adding a global reference yields 5.3% rate savings over the context-only model at low bitrates.
• Incorporating the confidence map U further improves the performance of the reference model.
• Replacing GDN with the proposed GSDN provides an additional ~2.0% rate savings.
• The approach achieves up to 21% BD-rate savings relative to BPG at low bitrates.
• The method achieves progressive improvements when combining local context, global reference, and hyperprior without increasing overall model complexity.