[Paper Review] Self-supervised Low Light Image Enhancement and Denoising

Yu Zhang, Xiaoguang Di+4 more|arXiv (Cornell University)|Mar 1, 2021

Image Enhancement TechniquesComputer Science32 references18 citations

TL;DR

This paper proposes a self-supervised two-stage deep learning framework for low-light image enhancement and denoising, using an Image Contrast Enhancement Network (ICE-Net) and a Re-Enhancement and Denoising Network (RED-Net). The method achieves state-of-the-art visual quality and noise reduction without paired training data by leveraging a Maximum Entropy-based Retinex model and noise independence assumptions, outperforming supervised methods in detail preservation and adaptability across devices.

ABSTRACT

This paper proposes a self-supervised low light image enhancement method based on deep learning, which can improve the image contrast and reduce noise at the same time to avoid the blur caused by pre-/post-denoising. The method contains two deep sub-networks, an Image Contrast Enhancement Network (ICE-Net) and a Re-Enhancement and Denoising Network (RED-Net). The ICE-Net takes the low light image as input and produces a contrast enhanced image. The RED-Net takes the result of ICE-Net and the low light image as input, and can re-enhance the low light image and denoise at the same time. Both of the networks can be trained with low light images only, which is achieved by a Maximum Entropy based Retinex (ME-Retinex) model and an assumption that noises are independently distributed. In the ME-Retinex model, a new constraint on the reflectance image is introduced that the maximum channel of the reflectance image conforms to the maximum channel of the low light image and its entropy should be the largest, which converts the decomposition of reflectance and illumination in Retinex model to a non-ill-conditioned problem and allows the ICE-Net to be trained with a self-supervised way. The loss functions of RED-Net are carefully formulated to separate the noises and details during training, and they are based on the idea that, if noises are independently distributed, after the processing of smoothing filters (\eg mean filter), the gradient of the noise part should be smaller than the gradient of the detail part. It can be proved qualitatively and quantitatively through experiments that the proposed method is efficient.

Motivation & Objective

• To address the challenge of low-light image enhancement with simultaneous contrast improvement and noise reduction, especially in the absence of paired real-world training data.
• To eliminate reliance on supervised data by enabling self-supervised training using only low-light images.
• To preserve fine details and edges while effectively removing noise, avoiding blur from pre/post-denoising pipelines.
• To develop a method robust to diverse devices, lighting conditions, and image content without requiring retraining or labeled references.
• To improve generalization and real-world applicability by avoiding assumptions about natural image priors or exposure-based reference images.

Proposed method

• ICE-Net is trained via a Maximum Entropy-based Retinex (ME-Retinex) model that enforces the reflectance image’s maximum channel to match the input low-light image and maximizes its entropy, stabilizing the ill-posed decomposition of reflectance and illumination.
• The ME-Retinex model introduces a non-ill-conditioned constraint by linking reflectance’s maximum channel to the input image and maximizing its entropy, enabling self-supervised training.
• RED-Net performs re-enhancement and denoising using the output of ICE-Net and the original low-light image, leveraging the assumption that noise is independently distributed across pixels.
• Noise and detail separation in RED-Net is achieved by analyzing gradients after smoothing filters: noise gradients are smaller than detail gradients, allowing gradient-based weighting in loss functions.
• The loss functions for RED-Net are designed to amplify gradients of edges and details to enhance contrast, differing from prior methods that only preserve edges.
• Both networks are trained end-to-end with self-supervised loss functions derived from physical assumptions about noise and reflectance, avoiding the need for ground-truth images.

Experimental results

Research questions

• RQ1 Can low-light image enhancement and denoising be effectively trained without paired real-world reference images?
• RQ2 Can a self-supervised framework maintain high contrast and detail preservation while removing noise in low-light conditions?
• RQ3 How can noise and structural details be separated in the absence of ground-truth labels?
• RQ4 Does re-enhancing the output of a contrast-enhanced network improve denoising and detail recovery compared to post-processing?
• RQ5 Can the method generalize across different devices and lighting conditions without retraining?

Key findings

• The proposed method achieves a PSNR of 18.34 on the LOL dataset, outperforming KinD (17.64) and Retinex-Net (16.82), indicating superior noise suppression.
• On the LOL dataset, the method achieves an SSIM of 0.65, outperforming LIME (0.60), RRM (0.66), and Retinex-Net (0.57), showing better structural similarity.
• In visual comparisons, the method preserves text and fine structures in low-light images (e.g., bookshelves and facial details in shadows) better than LIME and Retinex-Net.
• The method achieves competitive results to supervised state-of-the-art methods like KinD, especially in edge sharpness and noise reduction, despite being self-supervised.
• The method shows limitations in NIQE (score of 4.33 on LOL), indicating that enhanced images deviate from natural image statistics, likely due to lack of natural image priors in loss functions.
• The framework successfully avoids blur from pre-/post-denoising pipelines and maintains high-quality enhancement even under non-uniform illumination and high noise levels.