[论文解读] Implicit Neural Representation for Cooperative Low-light Image Enhancement

Shuzhou Yang, Moxuan Ding+3 more|arXiv (Cornell University)|Mar 21, 2023

Image Enhancement Techniques被引用 9

一句话总结

NeRCo 引入基于隐式神经表示的框架，归一化降解、使用文本引导的语义监督，并以自监督的双环方式训练，以实现鲁棒且感知上令人愉悦的低光增强。

ABSTRACT

The following three factors restrict the application of existing low-light image enhancement methods: unpredictable brightness degradation and noise, inherent gap between metric-favorable and visual-friendly versions, and the limited paired training data. To address these limitations, we propose an implicit Neural Representation method for Cooperative low-light image enhancement, dubbed NeRCo. It robustly recovers perceptual-friendly results in an unsupervised manner. Concretely, NeRCo unifies the diverse degradation factors of real-world scenes with a controllable fitting function, leading to better robustness. In addition, for the output results, we introduce semantic-orientated supervision with priors from the pre-trained vision-language model. Instead of merely following reference images, it encourages results to meet subjective expectations, finding more visual-friendly solutions. Further, to ease the reliance on paired data and reduce solution space, we develop a dual-closed-loop constrained enhancement module. It is trained cooperatively with other affiliated modules in a self-supervised manner. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NeRCo. Our code is available at https://github.com/Ysz2022/NeRCo.

研究动机与目标

• 解决现实世界低光图像中不可预测的亮度降解与噪声。
• 缩小指标友好型增强与感知友好型增强之间的差距。
• 通过自监督的协同训练减少对成对数据的依赖。
• 结合来自视觉-语言先验的语义引导以提升视觉质量。

提出的方法

• 使用可控拟合函数的 Neural Representation Normalization (NRN) 来归一化降解。
• 使用 Mask Extractor (ME) 与 Collaborative Attention Module 指导区域感知增强。
• 采用 Text-driven Appearance Discriminator (TAD)，结合基于 CLIP 的文本与图像先验进行语义与外观监督。
• 通过前向（增强）和后向（降解）映射的双环协同框架进行训练以实现无监督学习。
• 施加循环一致性与协作损失以约束解空间并提高稳定性。

实验结果

研究问题

• RQ1 隐式神经表示是否能将现实世界的低光降解归一化，从而促进增强？
• RQ2 多模态（视觉-语言）监督是否在保持指标性能的同时提升感知质量？
• RQ3 双环自监督框架在无成对数据的情况下是否能实现具有竞争力的结果？
• RQ4 语义先验对增强图像的颜色、纹理与整体真实感有何影响？

主要发现

• NeRCo 在多个基准数据集上在全参考和无参考指标上均具备与现有方法相竞争的性能。
• 该方法常常超过 some supervised approaches，强调多模态监督与归一化的优点。
• 在无成对数据设置下，具有循环一致性的双环协作对抗训练策略实现了鲁棒训练。
• 文本驱动的监督有助于使输出与感知期望对齐，同时保持定量质量。
• 基于神经表示的归一化降低了降解的变异性并简化后续增强任务。