[Paper Review] Joint Global and Local Hierarchical Priors for Learned Image Compression

Jun-Hyuk Kim, Byeongho Heo+1 more|arXiv (Cornell University)|Dec 8, 2021

Advanced Data Compression Techniques4 citations

TL;DR

This paper proposes Informer, a novel learned entropy model for image compression that leverages both global and local hierarchical priors via a content-dependent attention mechanism. By combining a global hyperprior using cross-attention and a local hyperprior via 1×1 convolutions, Informer achieves state-of-the-art rate-distortion performance on Kodak and Tecnick datasets while avoiding the quadratic computational complexity of prior global methods.

ABSTRACT

Recently, learned image compression methods have outperformed traditional hand-crafted ones including BPG. One of the keys to this success is learned entropy models that estimate the probability distribution of the quantized latent representation. Like other vision tasks, most recent learned entropy models are based on convolutional neural networks (CNNs). However, CNNs have a limitation in modeling long-range dependencies due to their nature of local connectivity, which can be a significant bottleneck in image compression where reducing spatial redundancy is a key point. To overcome this issue, we propose a novel entropy model called Information Transformer (Informer) that exploits both global and local information in a content-dependent manner using an attention mechanism. Our experiments show that Informer improves rate--distortion performance over the state-of-the-art methods on the Kodak and Tecnick datasets without the quadratic computational complexity problem. Our source code is available at https://github.com/naver-ai/informer.

Motivation & Objective

• To address the limitation of CNN-based entropy models in capturing long-range spatial dependencies due to local receptive fields.
• To overcome the content-agnostic nature of convolutional operations that treat dissimilar image regions uniformly within a receptive field.
• To design an entropy model that effectively integrates both global and local dependencies in a content-dependent manner for improved compression efficiency.
• To achieve superior rate-distortion performance without incurring quadratic computational complexity, unlike prior global attention-based models.

Proposed method

• Proposes a new entropy model, Informer, which uses a cross-attention mechanism to generate a global hyperprior by attending to different regions of the quantized latent representation.
• Introduces a local hyperprior via 1×1 convolutional layers to specialize in local dependencies and prevent global tokens from overfitting to local patterns.
• Employs a joint autoregressive and hierarchical prior framework, where the context prior is updated using both local and global hyperpriors.
• Uses masked multi-head self-attention in the global hyperprior to model long-range dependencies in a content-aware fashion.
• Designs a dual-branch architecture where the global and local hyperpriors are processed separately and then fused before being used in the entropy estimation network.
• Employs a learnable global token mechanism with a variable number of tokens (N), allowing adaptive modeling of dependencies based on bit rate requirements.

Figure 1 : Overview of the proposed method. Our Informer is a learned entropy model capturing global dependencies in a content-dependent manner using the attention mechanism [ 46 ] .

Experimental results

Research questions

• RQ1 Can a content-dependent attention mechanism effectively model long-range spatial dependencies in the latent representation of image compression?
• RQ2 Does combining global and local hyperpriors improve rate-distortion performance compared to using only one type of prior?
• RQ3 Can the proposed method achieve better performance than state-of-the-art global attention-based models without suffering from quadratic computational complexity?
• RQ4 How does the number of global tokens affect the rate-distortion trade-off in different bit rate regimes?
• RQ5 Is the attention mechanism's superiority in image compression due to its architecture or due to the specific design of joint global and local priors?

Key findings

• Informer achieves a rate of 0.2763 bpp and PSNR of 31.3254 dB at λ = 0.0067 on the Kodak dataset, outperforming the baseline and global reference model.
• At λ = 0.0483, Informer achieves 0.8514 bpp and 36.9739 dB PSNR, again surpassing the global context model (0.8718 bpp, 36.9134 dB) and demonstrating consistent gains across bit rate levels.
• Using four global tokens yields optimal performance at low bit rates, while eight tokens are better at higher bit rates, indicating adaptive dependency modeling based on residual redundancy.
• Visualization of attention maps confirms that global tokens attend to distinct, content-relevant image regions (e.g., bricks vs. window), validating content-dependent global modeling.
• The ablation study shows that updating the context prior with the global hyperprior is more effective than updating the local hyperprior, indicating the global prior's stronger modeling capacity.
• Informer avoids the quadratic computational complexity of prior global models while maintaining or improving performance, making it more practical for real-world deployment.

(a) Hyperprior [ 8 ]