[论文解读] Selective compression learning of latent representations for variable-rate image compression

Jooyoung Lee, Seyoon Jeong+1 more|arXiv (Cornell University)|Nov 8, 2022

Advanced Image and Video Retrieval Techniques被引用 9

一句话总结

本文提出选择性表征压缩（Selective Compression of Representations, SCR），一种轻量级、端到端可训练的神经网络图像可变比特率压缩方法。该方法基于内容相关的重要性图与质量自适应调整曲线，自适应地选择潜在表征元素进行熵编码。该方法在极低开销下实现了与独立训练模型相当的压缩效率，并通过插值实现连续比特率控制，同时将解码时间最多减少25.71%。

ABSTRACT

Recently, many neural network-based image compression methods have shown promising results superior to the existing tool-based conventional codecs. However, most of them are often trained as separate models for different target bit rates, thus increasing the model complexity. Therefore, several studies have been conducted for learned compression that supports variable rates with single models, but they require additional network modules, layers, or inputs that often lead to complexity overhead, or do not provide sufficient coding efficiency. In this paper, we firstly propose a selective compression method that partially encodes the latent representations in a fully generalized manner for deep learning-based variable-rate image compression. The proposed method adaptively determines essential representation elements for compression of different target quality levels. For this, we first generate a 3D importance map as the nature of input content to represent the underlying importance of the representation elements. The 3D importance map is then adjusted for different target quality levels using importance adjustment curves. The adjusted 3D importance map is finally converted into a 3D binary mask to determine the essential representation elements for compression. The proposed method can be easily integrated with the existing compression models with a negligible amount of overhead increase. Our method can also enable continuously variable-rate compression via simple interpolation of the importance adjustment curves among different quality levels. The extensive experimental results show that the proposed method can achieve comparable compression efficiency as those of the separately trained reference compression models and can reduce decoding time owing to the selective compression. The sample codes are publicly available at https://github.com/JooyoungLeeETRI/SCR.

研究动机与目标

• 解决现有学习型图像压缩方法模型复杂度过高、需为每个目标比特率单独训练的问题。
• 通过避免增加额外的网络模块或层，降低可变比特率压缩中的计算与内存开销。
• 通过重要性调整曲线的插值，实现在无需重训练的情况下实现连续比特率压缩。
• 通过选择性地对潜在表征进行熵编码，在显著减少解码时间的同时保持或提升编码效率。
• 提供即插即用的解决方案，与现有压缩架构兼容，仅需极少的架构修改。

提出的方法

• 通过单个1×1卷积层生成一个3D重要性图，表示与目标质量无关的潜在表征元素的内容相关重要性。
• 针对每个目标质量等级，利用学习到的、通道独立的重要性调整曲线对3D重要性图进行调整，以反映特定质量下的优先级。
• 通过舍入操作对调整后的重要性图进行阈值化处理，生成一个3D二值掩码，以确定哪些潜在表征元素需进行熵编码。
• 通过仅增加少量参数（即1×1卷积和每个质量等级的若干条曲线）与现有模型无缝集成。
• 在推理阶段通过插值离散质量等级之间的调整曲线，实现连续比特率支持。
• 整个系统端到端训练，联合优化选择性压缩与自适应量化组件。

实验结果

研究问题

• RQ1 单个神经网络模型能否在不增加模型复杂度的前提下，实现与独立训练模型相当的编码效率，从而实现可变比特率图像压缩？
• RQ2 如何使潜在表征的选择性压缩适应不同的目标质量等级，同时保持高编码效率？
• RQ3 所提出的方法能否通过学习曲线的插值实现在无需重训练情况下的连续比特率压缩？
• RQ4 选择性压缩对解码时间有何影响？其速度能否超越现有轻量级可变比特率方法？
• RQ5 该方法引入的额外模型开销有多大？是否可被视为对现有压缩框架的即插即用型增强？

主要发现

• 与无选择性压缩的变体相比，SCR在Hyperprior和Mean-scale模型上分别将平均解码时间减少了11.28%和8.32%。
• 对于Context-based模型，SCR相比其非选择性变体将解码时间减少了25.71%，相比参考模型在q=8.0时减少了21.53%。
• 在所有目标质量等级下，该方法的编码效率与独立训练的参考模型相当，PSNR和BD-rate指标均表现优异。
• Hyperprior模型的参数仅增加0.59%（0.26 MB），Mean-scale和Context模型也表现出类似的极小开销。
• 通过在离散质量等级之间对重要性调整曲线进行非线性插值，该方法实现了稳定的连续比特率压缩。
• 由于选择性处理，编码时间也显著减少，相关结果见补充材料。