[Paper Review] Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression

Huairui Wang, Nianxiang Fu+2 more|arXiv (Cornell University)|Aug 17, 2023

Advanced Data Compression TechniquesComputer Science3 citations

TL;DR

This paper proposes Dynamic Kernel-based Image Compression (DKIC), a learned image compression framework that uses dynamic kernel-based adaptive spatial aggregation to enhance receptive field flexibility and content-aware feature aggregation. By integrating Lite Deformable Convolution (LDCN) with a generalized coarse-to-fine entropy model and asymmetric spatial-channel modeling, DKIC achieves state-of-the-art rate-distortion performance with moderate model complexity, outperforming VTM-12.1 and SOTA methods on standard benchmarks.

ABSTRACT

Learned image compression methods have shown superior rate-distortion performance and remarkable potential compared to traditional compression methods. Most existing learned approaches use stacked convolution or window-based self-attention for transform coding, which aggregate spatial information in a fixed range. In this paper, we focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. With the adaptive aggregation strategy and the sharing weights mechanism, our method can achieve promising transform capability with acceptable model complexity. Besides, according to the recent progress of entropy model, we define a generalized coarse-to-fine entropy model, considering the coarse global context, the channel-wise, and the spatial context. Based on it, we introduce dynamic kernel in hyper-prior to generate more expressive global context. Furthermore, we propose an asymmetric spatial-channel entropy model according to the investigation of the spatial characteristics of the grouped latents. The asymmetric entropy model aims to reduce statistical redundancy while maintaining coding efficiency. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.

Motivation & Objective

• To overcome the fixed-range spatial aggregation limitation of standard convolution and window-based attention in learned image compression.
• To enhance receptive field adaptivity by enabling content-conditioned, dynamic kernel offsets for improved feature representation.
• To design a generalized coarse-to-fine entropy model that integrates global, channel-wise, and spatial context for better probability estimation.
• To reduce statistical redundancy through an asymmetric spatial-channel entropy model tailored to grouped latent distributions.
• To achieve superior rate-distortion performance while maintaining acceptable model complexity and inference speed.

Proposed method

• Introduces a dynamic kernel mechanism based on Lite Deformable Convolution (LDCN) that learns content-adaptive kernel offsets and group-shared weights for flexible spatial aggregation.
• Employs a residual bottleneck block with post-normalization and dynamic kernel layers to improve non-linear representation learning.
• Proposes a generalized coarse-to-fine entropy model that jointly models global context, channel-wise dependencies, and spatial context for improved distribution estimation.
• Designs an asymmetric spatial-channel entropy model that reduces redundancy by modeling spatial and channel contexts differently based on latent group characteristics.
• Utilizes local attribution maps (LAM) with path integral gradients to visualize and analyze effective receptive fields (ERF) of different methods.
• Employs a two-stage training strategy with joint autoregressive entropy modeling and progressive context refinement to balance accuracy and speed.

Experimental results

Research questions

• RQ1 Can dynamic kernel-based spatial aggregation improve the receptive field adaptivity and content-aware feature learning in learned image compression?
• RQ2 How does the integration of coarse-to-fine entropy modeling enhance probability estimation and reduce redundancy in latent representations?
• RQ3 To what extent does the asymmetric spatial-channel entropy model reduce statistical redundancy while preserving coding efficiency?
• RQ4 How does the proposed method compare in rate-distortion performance and computational cost against state-of-the-art learned compression models?
• RQ5 What is the impact of dynamic kernel design on effective receptive field shape and feature relevance in reconstruction?

Key findings

• DKIC achieves the best rate-distortion performance among all compared methods on the Kodak, CLIC, and other benchmarks, outperforming VTM-12.1 and SOTA approaches like STF and InvComp.
• The effective receptive field (ERF) visualization confirms that DKIC captures more relevant, content-conditioned regions—forming oval-like, focused fields—while avoiding irrelevant pixels, unlike fixed-kernel or window-attention methods.
• DKIC achieves a coding time of ~150ms per 768×512 image on a single RTX 3090, outperforming autoregressive methods like Cheng2020 and InvComp in speed while maintaining superior RD performance.
• The proposed asymmetric spatial-channel entropy model reduces redundancy and enables high coding efficiency, as validated through ablation studies on context modeling strategies.
• The post-normalization dynamic residual block design (LN after LDCN and MLP) yields the best performance, aligning with findings in other transformer and CNN architectures.
• DKIC has 53.3M parameters, making it more efficient than SwinT-ChARM (60.5M) and significantly more efficient than STF (99.8M), while still achieving superior RD performance.