[论文解读] LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement

Tao Wang, Kaihao Zhang+6 more|arXiv (Cornell University)|Jul 27, 2023

Image Enhancement Techniques被引用 8

一句话总结

LLDiffusion 将退化表示引入扩散式框架，用于联合学习低光退化与增强，在合成与真实数据集上达到最先进的 LLIE 性能。

ABSTRACT

Current deep learning methods for low-light image enhancement (LLIE) typically rely on pixel-wise mapping learned from paired data. However, these methods often overlook the importance of considering degradation representations, which can lead to sub-optimal outcomes. In this paper, we address this limitation by proposing a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process, resulting in improved image enhancement. Our proposed degradation-aware learning scheme is based on the understanding that degradation representations play a crucial role in accurately modeling and capturing the specific degradation patterns present in low-light images. To this end, First, a joint learning framework for both image generation and image enhancement is presented to learn the degradation representations. Second, to leverage the learned degradation representations, we develop a Low-Light Diffusion model (LLDiffusion) with a well-designed dynamic diffusion module. This module takes into account both the color map and the latent degradation representations to guide the diffusion process. By incorporating these conditioning factors, the proposed LLDiffusion can effectively enhance low-light images, considering both the inherent degradation patterns and the desired color fidelity. Finally, we evaluate our proposed method on several well-known benchmark datasets, including synthetic and real-world unpaired datasets. Extensive experiments on public benchmarks demonstrate that our LLDiffusion outperforms state-of-the-art LLIE methods both quantitatively and qualitatively. The source code and pre-trained models are available at https://github.com/TaoWangzj/LLDiffusion.

研究动机与目标

• Motivate LLIE to leverage degradation representations rather than solely pixel-wise mappings.
• Propose a joint learning framework to capture low-light degradation from data.
• Develop a dynamic degradation-aware diffusion module to integrate degradation with image priors during enhancement.
• Demonstrate superior quantitative and qualitative performance on synthetic and real-world LLIE benchmarks.

提出的方法

• Introduce a latent map encoder to extract degradation representations from low-light inputs.
• Train a degradation generation network (DGNET) to simulate degradation from normal-light to low-light images.
• Develop a dynamic degradation-aware diffusion module (DDDM) that conditions enhancement on degradation representations and invariant color maps.
• Jointly train DGNET and DDDM with a combined loss to learn degradation representations during a degradation-aware learning phase.
• Freeze the encoder after pre-training and fine-tune the diffusion module for enhancement conditioned on color maps and degradation representations.

实验结果

研究问题

• RQ1 Can degradation representations improve diffusion-based LLIE by capturing real-world degradation patterns?
• RQ2 Does a dynamic, degradation-aware diffusion module improve color fidelity and perceptual quality in LLIE?
• RQ3 How does joint learning of degradation representation and enhancement compare to traditional end-to-end LLIE methods?
• RQ4 What is the impact of invariant color maps as image priors in diffusion-based LLIE?

主要发现

Method	PSNR	SSIM	LPIPS
BIMEF	13.88	0.595	0.326
NPE	16.97	0.484	0.405
LIME	16.76	0.445	0.395
MF	16.97	0.508	0.380
SRIE	11.86	0.495	0.257
RetinexNet	16.77	0.560	0.474
KinD	20.87	0.790	0.170
KinD++	21.30	0.820	0.160
Zero-DCE	14.86	0.562	0.335
RUAS	18.23	0.720	0.350
EnlightenGAN	17.48	0.652	0.322
Uformer	18.55	0.721	0.321
Restormer	22.37	0.816	0.141
LLFormer	23.65	0.816	0.171
MIRNet	24.14	0.830	0.131
LLDiffusion	24.65	0.843	0.075

• LLDiffusion achieves superior PSNR, SSIM, and LPIPS on LOL and VE-LOL datasets compared to 15 SOTA methods, including MIRNet and LLFormer.
• On LOL, LLDiffusion records PSNR 24.65, SSIM 0.843, LPIPS 0.075 (best among evaluated methods).
• Qualitatively, LLDiffusion produces brighter, more color-faithful, and artifact-free enhancements with clearer textures.
• The method generalizes to real-world unpaired datasets and a Real World Test (RWT) dataset, demonstrating robust performance across synthetic and real degradations.
• A two-stage training scheme (joint degradation learning and diffusion-based enhancement) with a pre-trained encoder yields effective degradation representations for LLIE.