[論文レビュー] Implicit Neural Representation for Cooperative Low-light Image Enhancement

Shuzhou Yang, Moxuan Ding+3 more|arXiv (Cornell University)|Mar 21, 2023

Image Enhancement Techniques被引用数 9

ひとこと要約

NeRCo は、劣化を正規化する潜在ニューラル表現ベースのフレームワークを導入し、テキストでガイドされた意味論的監視を用い、自己教師付きのデュアルループ学習方式で頑健かつ知覚的に心地よい低照度強化を実現します。

ABSTRACT

The following three factors restrict the application of existing low-light image enhancement methods: unpredictable brightness degradation and noise, inherent gap between metric-favorable and visual-friendly versions, and the limited paired training data. To address these limitations, we propose an implicit Neural Representation method for Cooperative low-light image enhancement, dubbed NeRCo. It robustly recovers perceptual-friendly results in an unsupervised manner. Concretely, NeRCo unifies the diverse degradation factors of real-world scenes with a controllable fitting function, leading to better robustness. In addition, for the output results, we introduce semantic-orientated supervision with priors from the pre-trained vision-language model. Instead of merely following reference images, it encourages results to meet subjective expectations, finding more visual-friendly solutions. Further, to ease the reliance on paired data and reduce solution space, we develop a dual-closed-loop constrained enhancement module. It is trained cooperatively with other affiliated modules in a self-supervised manner. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NeRCo. Our code is available at https://github.com/Ysz2022/NeRCo.

研究の動機と目的

• Address unpredictable brightness degradation and noise in real-world low-light images.
• Bridge the gap between metric-favorable and perceptual-friendly enhancements.
• Reduce reliance on paired data through self-supervised cooperative training.
• Incorporate semantic guidance from vision-language priors to improve visual quality.

提案手法

• Normalize degradation using Neural Representation Normalization (NRN) with a controllable fitting function.
• Use a Mask Extractor (ME) and Collaborative Attention Module to guide region-aware enhancement.
• Employ a Text-driven Appearance Discriminator (TAD) that combines CLIP-based text and image priors for semantic and appearance supervision.
• Train a dual-loop cooperative framework with forward (enhance) and backward (degrade) mappings to enable unsupervised learning.
• Impose cycle-consistency and cooperative losses to constrain the solution space and improve stability.

実験結果

リサーチクエスチョン

• RQ1 Can implicit neural representations normalize real-world low-light degradation to ease enhancement?
• RQ2 Does multimodal (vision-language) supervision improve perceptual quality while maintaining metric performance?
• RQ3 Can a dual-loop, self-supervised framework achieve competitive results without paired data?
• RQ4 What is the impact of semantic priors on color, texture, and overall realism in enhanced images?

主な発見

• NeRCo achieves competitive to state-of-the-art performance on several benchmarks in both full-reference and no-reference metrics.
• The method often surpasses some supervised approaches, underscoring the benefit of multimodal supervision and normalization.
• A dual-loop cooperative adversarial strategy with cycle-consistency yields robust training under unpaired data settings.
• Text-driven supervision helps align outputs with perceptual expectations while maintaining quantitative quality.
• A neural representation-based normalization reduces degradation variability and eases subsequent enhancement tasks.