[论文解读] Low-light Image Enhancement by Retinex Based Algorithm Unrolling and Adjustment

Xinyi Liu, Qi Xie+3 more|arXiv (Cornell University)|Feb 12, 2022

Image Enhancement Techniques被引用 4

一句话总结

本文提出了一种新颖的深度学习框架，通过将基于Retinex的优化算法展开为神经网络，结合数据驱动先验与传统图像先验，实现低光照图像增强。该方法采用受算法展开启发的分解网络和双路调整模块，分别处理全局与局部亮度，通过自监督微调策略实现无需人工调参的最先进性能。

ABSTRACT

Motivated by their recent advances, deep learning techniques have been widely applied to low-light image enhancement (LIE) problem. Among which, Retinex theory based ones, mostly following a decomposition-adjustment pipeline, have taken an important place due to its physical interpretation and promising performance. However, current investigations on Retinex based deep learning are still not sufficient, ignoring many useful experiences from traditional methods. Besides, the adjustment step is either performed with simple image processing techniques, or by complicated networks, both of which are unsatisfactory in practice. To address these issues, we propose a new deep learning framework for the LIE problem. The proposed framework contains a decomposition network inspired by algorithm unrolling, and adjustment networks considering both global brightness and local brightness sensitivity. By virtue of algorithm unrolling, both implicit priors learned from data and explicit priors borrowed from traditional methods can be embedded in the network, facilitate to better decomposition. Meanwhile, the consideration of global and local brightness can guide designing simple yet effective network modules for adjustment. Besides, to avoid manually parameter tuning, we also propose a self-supervised fine-tuning strategy, which can always guarantee a promising performance. Experiments on a series of typical LIE datasets demonstrated the effectiveness of the proposed method, both quantitatively and visually, as compared with existing methods.

研究动机与目标

• 解决现有基于Retinex的深度学习方法在利用传统先验方面不足，且调整模块性能欠佳的问题。
• 通过算法展开，将隐式数据先验与显式传统先验嵌入图像分解过程，提升分解效果。
• 设计一种简单而高效的调整网络，同时考虑全局光照与局部亮度敏感性。
• 通过引入自监督微调策略，消除人工超参数调优的需要，同时保持高性能。

提出的方法

• 通过基于Retinex的优化过程的算法展开构建分解网络，实现数据驱动先验与传统先验的融合。
• 调整网络由两条并行分支组成：一条用于全局亮度调整，另一条用于局部亮度敏感性，从而提升细节保留能力。
• 该框架采用自监督微调策略，自动适应网络参数，无需配对的真实世界数据。
• 优化过程采用拟牛顿法建模，通过海塞矩阵近似实现高效且稳定的训练。
• 该方法将图像增强建模为包含反射率、光照与梯度先验的最小化问题，通过迭代更新求解。
• 最终输出通过元素级运算将增强后的反射率层与调整后的光照层结合，以保持空间一致性。

Figure 1: Illustration of the proposed LIE framework.

实验结果

研究问题

• RQ1 算法展开能否有效整合传统图像先验与深度学习，用于低光照图像增强？
• RQ2 如何设计一种简单而高效的调整模块，以同时处理低光照图像中的全局与局部亮度？
• RQ3 自监督微调策略是否能消除对人工超参数调优的需求，同时保持高性能？
• RQ4 将数据驱动先验与传统方法中的显式先验相结合，是否能带来更优的分解与增强效果？

主要发现

• 所提方法在多个标准低光照图像增强基准上均达到最先进性能，无论在定量还是定性指标上均表现优异。
• 自监督微调策略在无需真实标签或人工调优的情况下，持续提升各数据集上的性能表现。
• 通过算法展开整合数据驱动先验与传统先验，实现了更准确且物理上合理的光照与反射率分解。
• 双路调整网络在细节恢复与色彩保真度方面显著优于单路或复杂网络结构的调整模块。
• 在基准数据集上，该方法在PSNR、SSIM与LPIPS指标上均优于现有基于Retinex的深度学习模型。

(a) The entire decomposition network.