[논문 리뷰] An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization

Jooyoung Lee, Seung-Hyun Cho외 1명|arXiv (Cornell University)|2019. 12. 30.

Advanced Data Compression Techniques참고 문헌 22인용 수 40

한 줄 요약

JointIQ-Net은 가우시안 혼합 모델과 글로벌 컨텍스트를 사용하여 엔트로피 최소화를 개선한 cascade 구조로 이미지 압축과 품질 개선 모듈을 함께 최적화합니다. PSNR과 MS-SSIM에서 VVC Intra를 능가하는 최신 결과를 달성합니다.

ABSTRACT

Recently, learned image compression methods have been actively studied. Among them, entropy-minimization based approaches have achieved superior results compared to conventional image codecs such as BPG and JPEG2000. However, the quality enhancement and rate-minimization are conflictively coupled in the process of image compression. That is, maintaining high image quality entails less compression and vice versa. However, by jointly training separate quality enhancement in conjunction with image compression, the coding efficiency can be improved. In this paper, we propose a novel joint learning scheme of image compression and quality enhancement, called JointIQ-Net, as well as entropy model improvement, thus achieving significantly improved coding efficiency against the previous methods. Our proposed JointIQ-Net combines an image compression sub-network and a quality enhancement sub-network in a cascade, both of which are end-to-end trained in a combined manner within the JointIQ-Net. Also the JointIQ-Net benefits from improved entropy-minimization that newly adopts a Gussian Mixture Model (GMM) and further exploits global context to estimate the probabilities of latent representations. In order to show the effectiveness of our proposed JointIQ-Net, extensive experiments have been performed, and showed that the JointIQ-Net achieves a remarkable performance improvement in coding efficiency in terms of both PSNR and MS-SSIM, compared to the previous learned image compression methods and the conventional codecs such as VVC Intra (VTM 7.1), BPG, and JPEG2000. To the best of our knowledge, this is the first end-to-end optimized image compression method that outperforms VTM 7.1 (Intra), the latest reference software of the VVC standard, in terms of the PSNR and MS-SSIM.

연구 동기 및 목표

• Motivate improved coding efficiency by jointly optimizing image compression and quality enhancement.
• Develop an end-to-end framework that can integrate any quality-enhancement network in cascade with compression.
• Improve entropy modeling using Gaussian Mixture Models and global contextual information.
• Demonstrate that joint training yields better rate-distortion performance than separate training.

제안 방법

• Propose JointIQ-Net: a cascade of an image compression sub-network and a quality-enhancement sub-network (GRDN) trained jointly end-to-end.
• Adopt an improved entropy model with Gaussian Mixture Model (GMM) priors for the latent representation y_hat, estimated via a context-aware model estimator f.
• Incorporate a global context c''' for estimating GMM parameters, using a dedicated global-context extraction module with MPRM refinements.
• Utilize a hyperprior z_hat and an autoregressive y_hat|z_hat with prior modeling, plus a density-convolution trick to handle quantization.
• Train with a combined loss L = R + lambda D, where R is rate (via the learned priors) and D is distortion measured against a final enhanced output.
• Enable flexible integration of any quality-enhancement network; in experiments, GRDN is cascaded with the image compression network to form JointIQ-Net.

실험 결과

연구 질문

• RQ1 Can joint end-to-end optimization of image compression and quality enhancement yield better rate-distortion performance than separately trained components?
• RQ2 Does incorporating a Gaussian Mixture Model prior with global context improve entropy estimation and coding efficiency?
• RQ3 How does the joint scheme perform against VVC Intra, BPG, JPEG2000, and prior learned methods in PSNR and MS-SSIM?
• RQ4 What is the relative contribution of GRDN, global context, MPRM, and GMM to overall performance?
• RQ5 Does the proposed global-context mechanism effectively capture non-local dependencies to improve coding efficiency?

주요 결과

• JointIQ-Net outperforms prior learned methods and conventional codecs in PSNR and MS-SSIM on Kodak PhotoCD tests.
• The method reportedly surpasses VVC Intra (VTM 7.1) in both PSNR and MS-SSIM, marking the first learned image compression approach to do so.
• GMM-based priors with an enhanced estimator and global context provide coding gains over single-Gaussian models.
• A cascade with GRDN as the quality-enhancement module yields the best performance among tested configurations.
• Ablation studies show significant gains from GRDN and GMM; global context yields additional improvements, while MPRM aids higher bitrates.