[论文解读] EnlightenGAN: Deep Light Enhancement without Paired Supervision

Yifan Jiang, Xinyu Gong+7 more|arXiv (Cornell University)|Jun 17, 2019

Image Enhancement Techniques参考文献 53被引用 205

一句话总结

EnlightenGAN 是一种用于低光图像增强的无监督 GAN，通过使用未配对的低光/正常光图像、全局-局部判别器、自我正则化感知损失和自我正则化注意力，在没有配对数据的情况下产生照片级真实感结果。

ABSTRACT

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. The code is available at \url{https://github.com/yueruchen/EnlightenGAN}

研究动机与目标

• Motivate robust low-light enhancement without requiring paired low-/normal-light image pairs.
• Propose an unsupervised GAN framework that generalizes across diverse real-world scenes.
• Develop mechanisms to preserve content and texture while correcting illumination.
• Demonstrate superior visual quality and user preference compared to state-of-the-art methods.
• Show adaptability of EnlightenGAN to real-world domain shifts without paired supervision.

提出的方法

• Use an attention-guided U-Net as the generator.
• Employ a dual-discriminator setup (global and local PatchGAN-based discriminators).
• Adopt a relativistic LS-GAN formulation for the global discriminator and standard LS-GAN for the local discriminator.
• Introduce a self feature preserving loss that regularizes feature-space distance between the input low-light image and its enhanced output using VGG features.
• Incorporate a self-regularized attention mechanism based on the input illumination to modulate feature maps.
• Train with unpaired low-light and normal-light images without cycle-consistency, enabling faster training.

实验结果

研究问题

• RQ1 Can high-quality low-light enhancement be achieved without paired training data?
• RQ2 Do global and local discriminators jointly improve spatially varying illumination without introducing artifacts?
• RQ3 Does a self-regularized perceptual loss help preserve content in unpaired training?
• RQ4 Can an attention mechanism guided by input illumination further improve visual realism?

主要发现

• EnlightenGAN achieves superior visual quality and user preference over several baselines in subjective studies.
• The global-local discriminator structure helps prevent local over- or under-exposure artifacts.
• Self feature preserving loss provides meaningful perceptual regularization in an unpaired setting.
• Self-regularized attention improves consistency and realism across diverse scenes.
• The method demonstrates strong generalization and easy domain adaptation to real-world low-light data.
• Quantitative NIQE assessments and human evaluations favor EnlightenGAN on multiple test sets.