[Paper Review] Self-supervised Image Enhancement Network: Training with Low Light Images Only

Yu Zhang, Xiaoguang Di+2 more|arXiv (Cornell University)|Feb 26, 2020

Image Enhancement TechniquesComputer Science43 references91 citations

TL;DR

A self-supervised low-light image enhancement method based on a maximum entropy Retinex model that trains using only low-light images, achieving fast, realtime capable enhancement without paired data.

ABSTRACT

This paper proposes a self-supervised low light image enhancement method based on deep learning. Inspired by information entropy theory and Retinex model, we proposed a maximum entropy based Retinex model. With this model, a very simple network can separate the illumination and reflectance, and the network can be trained with low light images only. We introduce a constraint that the maximum channel of the reflectance conforms to the maximum channel of the low light image and its entropy should be largest in our model to achieve self-supervised learning. Our model is very simple and does not rely on any well-designed data set (even one low light image can complete the training). The network only needs minute-level training to achieve image enhancement. It can be proved through experiments that the proposed method has reached the state-of-the-art in terms of processing speed and effect.

Motivation & Objective

• Motivate improvements in low-light image enhancement without requiring paired normal-light data.
• Introduce a maximum entropy based Retinex model to separate reflectance and illumination.
• Design a simple CNN that enables self-supervised training with low-light images only.
• Demonstrate real-time performance and strong generalization across environments and devices.

Proposed method

• Formulate a maximum entropy based Retinex objective combining reconstruction, reflectance, and illumination losses with L1 norms.
• Constrain reflectance using the maximum channel and its histogram-equalized entropy.
• Adopt a structure-aware smoothness loss for illumination to preserve structure information.
• Implement a simple fully convolutional network that outputs reflectance and illumination via a Sigmoid at the end.
• Train the network in a self-supervised manner using only low-light images (even a single image) without paired data.
• Evaluate using objective metrics and compare against classic and state-of-the-art methods.

Experimental results

Research questions

• RQ1 Can low-light image enhancement be learned in a self-supervised way using only low-light images?
• RQ2 Does a maximum-entropy Retinex formulation yield good reflectance/illumination decomposition and enhanced images without reference data?
• RQ3 How does the proposed method compare to supervised and other unsupervised methods in terms of quality and speed?
• RQ4 What is the impact of training with single versus multiple low-light images on stability and artifacts?

Key findings

• The method achieves state-of-the-art processing speed and competitive enhancement quality compared to traditional and deep learning baselines.
• Training can be conducted with only low-light images, even a single image, achieving minute-level training and real-time performance.
• The proposed maximum-entropy Retinex model provides effective separation of reflectance and illumination with entropy-based constraints.
• Early stopping helps avoid noise and artifacts when training across many epochs.
• Quantitative results on the LOL dataset show competitive PSNR/SSIM and favorable qualitative results against several baselines.
• Single-image training remains feasible and artifact-free with adequate epochs, illustrating strong adaptability to new environments.