[Paper Review] SID-NISM: A Self-supervised Low-light Image Enhancement Framework

Lijun Zhang, Xiao Liu+2 more|arXiv (Cornell University)|Dec 16, 2020

Image Enhancement Techniques24 references4 citations

TL;DR

This paper proposes SID-NISM, a self-supervised low-light image enhancement framework that decomposes low-light images into reflectance, illumination, and noise components without requiring labeled training data. By leveraging a self-supervised image decomposition network (SID-Net) and a nonlinear illumination saturation mapping function (NISM), the method enhances image quality with fewer artifacts and improved naturalness, outperforming state-of-the-art methods in both subjective and objective evaluations, including a 33.39% mAP on object detection in low-light conditions.

ABSTRACT

When capturing images in low-light conditions, the images often suffer from low visibility, which not only degrades the visual aesthetics of images, but also significantly degenerates the performance of many computer vision algorithms. In this paper, we propose a self-supervised low-light image enhancement framework (SID-NISM), which consists of two components, a Self-supervised Image Decomposition Network (SID-Net) and a Nonlinear Illumination Saturation Mapping function (NISM). As a self-supervised network, SID-Net could decompose the given low-light image into its reflectance, illumination and noise directly without any prior training or reference image, which distinguishes it from existing supervised-learning methods greatly. Then, the decomposed illumination map will be enhanced by NISM. Having the restored illumination map, the enhancement can be achieved accordingly. Experiments on several public challenging low-light image datasets reveal that the images enhanced by SID-NISM are more natural and have less unexpected artifacts.

Motivation & Objective

• To address the limitations of supervised learning methods in low-light image enhancement that rely on large-scale paired datasets.
• To develop a self-supervised framework that enhances low-light images without requiring reference images or prior training.
• To improve image quality by decomposing images into reflectance, illumination, and noise components using Retinex theory.
• To preserve contrast and enhance brightness in low-light regions while minimizing unnatural artifacts.
• To evaluate the method’s effectiveness in downstream computer vision tasks, such as object detection.

Proposed method

• SID-NISM employs a self-supervised image decomposition network (SID-Net) that decomposes a low-light image and its histogram-equalized version into reflectance, smooth illumination, and noise maps.
• SID-Net uses a novel loss function combining reflectance and noise-specific terms with standard regularization to guide decomposition toward physically plausible results.
• The illumination map is enhanced via a nonlinear illumination saturation mapping (NISM) function that preserves contrast and brightens dark regions without overexposure.
• The final enhanced image is reconstructed by combining the denoised reflectance and the enhanced illumination map.
• The framework is trained end-to-end using only the input image and its histogram-equalized variant, eliminating the need for paired training data.
• The method is evaluated on public datasets including LOL and MEF, with comparisons to state-of-the-art methods using both objective metrics and human perception studies.

Experimental results

Research questions

• RQ1 Can a self-supervised framework achieve high-quality low-light image enhancement without relying on paired training data or prior knowledge?
• RQ2 How effective is the proposed SID-Net in decomposing low-light images into physically meaningful components (reflectance, illumination, noise) in a self-supervised manner?
• RQ3 To what extent does the NISM function improve illumination enhancement while preserving contrast and avoiding over-brightening?
• RQ4 How does the enhanced image quality impact downstream computer vision tasks like object detection?
• RQ5 How do the proposed loss terms for reflectance and noise contribute to the overall image quality and artifact reduction?

Key findings

• SID-NISM achieved the highest mAP of 33.39% on the ExDark dataset for object detection, outperforming the original low-light images (31.54%) and matching the best-performing method (LIME at 33.98%).
• On the LOL and MEF datasets, SID-NISM achieved superior results in PSNR, SSIM, GE, CE, GMI, GMG, and NIQE, indicating better overall image quality and naturalness.
• The inclusion of the novel $γ_{R}$ and $γ_{N}$ loss terms in SID-Net significantly improved color fidelity, gradient preservation, and noise suppression.
• Human perception studies showed that participants preferred SID-NISM outputs over competing methods, reporting fewer unexpected artifacts.
• The method achieved strong performance without any external training data, demonstrating the effectiveness of self-supervision in low-light enhancement.
• The framework outperformed existing state-of-the-art methods in both objective metrics and subjective evaluations, confirming its robustness and visual quality.