[论文解读] Learning a Virtual Codec Based on Deep Convolutional Neural Network to Compress Image

Lijun Zhao, Huihui Bai+2 more|arXiv (Cornell University)|Dec 16, 2017

Advanced Data Compression Techniques参考文献 7被引用 7

一句话总结

本文提出了一种端到端的深度学习图像压缩框架，将标准编解码器（如JPEG）与特征描述网络及后处理网络相结合，通过可学习的虚拟编解码器实现对不可微分量化步骤的梯度反向传播。该方法在极低比特率下实现了最先进性能，显著减少了压缩伪影，同时保持与现有编解码器的兼容性。

ABSTRACT

Although deep convolutional neural network has been proved to efficiently eliminate coding artifacts caused by the coarse quantization of traditional codec, it's difficult to train any neural network in front of the encoder for gradient's back-propagation. In this paper, we propose an end-to-end image compression framework based on convolutional neural network to resolve the problem of non-differentiability of the quantization function in the standard codec. First, the feature description neural network is used to get a valid description in the low-dimension space with respect to the ground-truth image so that the amount of image data is greatly reduced for storage or transmission. After image's valid description, standard image codec such as JPEG is leveraged to further compress image, which leads to image's great distortion and compression artifacts, especially blocking artifacts, detail missing, blurring, and ringing artifacts. Then, we use a post-processing neural network to remove these artifacts. Due to the challenge of directly learning a non-linear function for a standard codec based on convolutional neural network, we propose to learn a virtual codec neural network to approximate the projection from the valid description image to the post-processed compressed image, so that the gradient could be efficiently back-propagated from the post-processing neural network to the feature description neural network during training. Meanwhile, an advanced learning algorithm is proposed to train our deep neural networks for compression. Obviously, the priority of the proposed method is compatible with standard existing codecs and our learning strategy can be easily extended into these codecs based on convolutional neural network. Experimental results have demonstrated the advances of the proposed method as compared to several state-of-the-art approaches, especially at very low bit-rate.

研究动机与目标

• 解决在将深度神经网络与标准图像编解码器集成时，量化步骤不可微分的问题。
• 实现特征描述网络、标准编解码器与后处理网络的端到端训练。
• 通过减少块效应、模糊和振 ringing 伪影，提升极低比特率下的图像重建质量。
• 在提升压缩效率的同时，保持与JPEG等现有标准编解码器的兼容性。
• 开发一种训练策略，实现梯度在不可微分编解码器阶段的有效反向传播。

提出的方法

• 特征描述神经网络（FDNN）将原始图像压缩为低维表示，以减小数据量。
• 低维表示通过标准编解码器（如JPEG）处理，引入可控的失真与伪影。
• 后处理神经网络（ComCNN）通过去除块效应和模糊等压缩伪影来恢复图像。
• 引入虚拟编解码器神经网络，近似从特征描述到压缩图像的映射，从而实现从后处理网络到FDNN的梯度流动。
• 采用联合优化策略，将整体问题分解为三个子问题：FDNN训练、虚拟编解码器训练与ComCNN训练。
• 虚拟编解码器网络作为标准编解码器的替代品，使梯度能够通过不可微分的量化步骤。

实验结果

研究问题

• RQ1 当在特征编码器与后处理网络之间插入不可微分的标准编解码器时，是否可以实现深度学习框架的端到端训练？
• RQ2 在深度学习压缩流水线中，如何有效实现梯度通过标准编解码器量化阶段的反向传播？
• RQ3 所学习的虚拟编解码器在极低比特率下，能在多大程度上改善伪影去除与重建质量？
• RQ4 在极低压缩率下，该方法在PSNR与SSIM指标上与最先进方法相比表现如何？
• RQ5 特征表示（FDNN输出）的选择是否显著影响压缩伪影的分布及后续恢复质量？

主要发现

• 所提方法在所有对比方法中取得了最高的PSNR与SSIM得分，尤其在极低比特率下表现优异（例如bpp ≤ 0.25）。
• 在客观指标与视觉质量方面，该方法显著优于Jiang等人[35]，尤其在保留发丝与眼轮廓等精细细节方面表现更佳。
• 虚拟编解码器网络实现了从后处理网络到特征描述网络的有效梯度反向传播，这对模型的最优训练至关重要。
• 该框架与JPEG等标准编解码器完全兼容，可无缝集成至现有压缩流水线，无需修改。
• 视觉对比显示，与Foi[14]、BM3D[22]、DicTV[17]和CONCOLOR[18]相比，所提方法更有效地减少了块效应与模糊伪影。
• 与Jiang等人[35]相比，本方法在伪影分布上的差异归因于FDNN学习到的不同特征表示，从而在解码器端实现了更优的重建效果。