[论文解读] End-to-End Learned Image Compression with Quantized Weights and Activations

Heming Sun, Lu Yu+1 more|arXiv (Cornell University)|Nov 17, 2021

Video Coding and Compression Technologies参考文献 55被引用 5

一句话总结

该论文提出了一种端到端可学习图像压缩框架，采用8位定点数对权重和激活值进行量化，相较于BPG在MS-SSIM性能上表现更优，同时将内存使用量降低了75%。该方法引入了通道级非线性权重量化，结合剪裁微调方案，设计了动态范围优化的激活值码书，并采用均值移除量化以最小化量化误差和比特深度成本。

ABSTRACT

End-to-end Learned image compression (LIC) has reached the traditional hand-crafted methods such as BPG (HEVC intra) in terms of the coding gain. However, the large network size prohibits the usage of LIC on resource-limited embedded systems. This paper reduces the network complexity by quantizing both weights and activations. 1) For the weight quantization, we study different kinds of grouping and quantization scheme at first. A channel-wise non-linear quantization scheme is determined based on the coding gain analysis. After that, we propose a fine tuning scheme to clip the weights within a certain range so that the quantization error can be reduced. 2) For the activation quantization, we first propose multiple non-linear quantization codebooks with different maximum dynamic ranges. By selecting an optimal one through a multiplexer, the quantization range can be saturated to the greatest extent. In addition, we also exploit the mean-removed quantization for the analysis transform outputs in order to reduce the bit-width cost for the specific channel with the large non-zero mean. By quantizing each weight and activation element from 32-bit floating point to 8-bit fixed point, the memory cost for both weight and activation can be reduced by 75% with negligible coding performance loss. As a result, our quantized LIC can still outperform BPG in terms of MS-SSIM. To our best knowledge, this is the first work to give a complete analysis on the coding gain and the memory cost for a quantized LIC network, which validates the feasibility of the hardware implementation.

研究动机与目标

• 通过降低内存与计算成本，实现端到端可学习图像压缩（LIC）在资源受限嵌入式系统中的实际部署。
• 分析量化LIC网络中编码增益与内存成本之间的权衡，尤其针对硬件实现的考量。
• 为权重与激活值开发完整的量化策略，以保持高水平的压缩性能。
• 通过新颖的量化方案与微调技术，最小化量化误差与比特宽度成本。

提出的方法

• 提出一种通道级非线性权重量化方案，通过编码增益分析进行优化，以最小化失真。
• 引入权重剪裁微调方法，限制权重范围以减少量化后的量化误差。
• 设计四种具有不同动态范围的非线性激活值码书，通过多路复用器选择以最大化量化精度。
• 对分析变换输出应用均值移除量化，以减少大非零均值通道的比特宽度。
• 将权重与激活值从32位浮点数量化至8位定点数，使内存成本降低75%。
• 采用基于超先验的网络架构（hyperprior-5）作为基线，对所有层（包括超路径）应用量化。

实验结果

研究问题

• RQ1 在量化LIC网络中，权重的最佳分组与量化方案是什么，以最大化编码增益？
• RQ2 如何在不牺牲压缩性能的前提下，最小化低精度权重与激活表示中的量化误差？
• RQ3 动态范围自适应的激活量化能否提升可学习图像压缩的效率？
• RQ4 均值移除量化如何影响分析变换中的比特深度需求与整体内存成本？
• RQ5 8位定点数量化在降低内存与计算成本的同时，能在多大程度上保持编码性能？

主要发现

• 所提出的量化LIC在MS-SSIM上优于BPG，表明尽管采用8位量化，仍具备更优的主观质量。
• 与全精度模型相比，权重与激活值的内存消耗降低了75%，权重成本较[18]高出4.48%，但解码器成本低21%。
• 权重剪裁微调方案显著降低了量化误差，提升了量化后的编码增益。
• 采用四种具有动态范围选择的非线性激活码书，能够更好地饱和量化范围，提升精度。
• 均值移除量化降低了高均值通道的比特深度需求，有助于降低激活值的内存成本。
• 乘法与加法的能耗分别从3.7pJ与0.9pJ（32位浮点）降低至0.2pJ与0.03pJ（8位整数），表明计算资源显著节省。