[Paper Review] R2RNet: Low-light Image Enhancement via Real-low to Real-normal Network

Hai Jiang, Xuan Zhu+5 more|arXiv (Cornell University)|Jun 28, 2021

Image Enhancement Techniques53 references17 citations

TL;DR

R2RNet proposes a novel Retinex-based deep learning framework for low-light image enhancement that decomposes images into illumination and reflectance maps, denoises the reflectance, and enhances contrast using both spatial and frequency-domain features. It achieves state-of-the-art performance on benchmark datasets and significantly improves downstream face detection accuracy, enabled by the first large-scale real-world paired low/normal-light dataset (LSRW).

ABSTRACT

Images captured in weak illumination conditions could seriously degrade the image quality. Solving a series of degradation of low-light images can effectively improve the visual quality of images and the performance of high-level visual tasks. In this study, a novel Retinex-based Real-low to Real-normal Network (R2RNet) is proposed for low-light image enhancement, which includes three subnets: a Decom-Net, a Denoise-Net, and a Relight-Net. These three subnets are used for decomposing, denoising, contrast enhancement and detail preservation, respectively. Our R2RNet not only uses the spatial information of the image to improve the contrast but also uses the frequency information to preserve the details. Therefore, our model acheived more robust results for all degraded images. Unlike most previous methods that were trained on synthetic images, we collected the first Large-Scale Real-World paired low/normal-light images dataset (LSRW dataset) to satisfy the training requirements and make our model have better generalization performance in real-world scenes. Extensive experiments on publicly available datasets demonstrated that our method outperforms the existing state-of-the-art methods both quantitatively and visually. In addition, our results showed that the performance of the high-level visual task (i.e. face detection) can be effectively improved by using the enhanced results obtained by our method in low-light conditions. Our codes and the LSRW dataset are available at: https://github.com/abcdef2000/R2RNet.

Motivation & Objective

• Address the limitations of existing low-light enhancement methods that rely on synthetic datasets and fail to generalize to real-world scenes.
• Improve image quality by simultaneously enhancing contrast, suppressing noise, and preserving fine details in low-light conditions.
• Demonstrate that low-light image enhancement can significantly improve the performance of high-level vision tasks such as face detection.
• Develop a large-scale, real-world paired dataset (LSRW) to support training of deep learning models for low-light enhancement with better generalization.
• Integrate frequency-domain processing in the relighting stage to preserve image details while maintaining contrast and reducing artifacts.

Proposed method

• Proposes a three-subnet architecture: Decom-Net for separating input low-light images into illumination and reflectance maps based on Retinex theory.
• Employs Denoise-Net to suppress noise in the reflectance map using the illumination map as a spatial constraint, preserving structural integrity.
• Uses Relight-Net to enhance contrast and brightness by combining spatial features for illumination adjustment and frequency-domain features (via fast Fourier transform) for detail recovery.
• Introduces a frequency loss function to guide the Relight-Net in reconstructing high-frequency details without amplifying noise.
• Trains the entire network end-to-end using perceptual and reconstruction losses to ensure high visual quality and structural fidelity.
• Leverages a novel large-scale real-world paired dataset (LSRW) with 5,650 image pairs to improve model generalization and avoid domain shift from synthetic data.

Experimental results

Research questions

• RQ1 Can a deep learning model that combines spatial and frequency-domain processing achieve superior low-light image enhancement compared to existing methods?
• RQ2 Does training on real-world paired low/normal-light images lead to better generalization and performance on real-world low-light scenes than synthetic datasets?
• RQ3 To what extent can low-light image enhancement improve the accuracy of downstream high-level vision tasks such as face detection?
• RQ4 How effective is the proposed frequency loss in preserving image details during relighting without amplifying noise?
• RQ5 Can a three-stage network (decomposition, denoising, relighting) outperform end-to-end models in terms of contrast, noise suppression, and detail preservation?

Key findings

• R2RNet achieves the highest PSNR (20.207) and SSIM (0.816) on the LOL dataset, outperforming state-of-the-art methods including MBLLEN and EnlightenGAN.
• The ablation study confirms that the full R2RNet architecture with both perceptual and frequency losses achieves the best performance, with PSNR dropping by 2.242 when frequency loss is removed.
• Using R2RNet as a pre-processing step increases the average precision (AP) of face detection from 17.12% (without enhancement) to 33.98% with DSFD and 25.97% with RetinaFace on the DARK FACE dataset.
• The LSRW dataset, containing 5,650 real-world paired low/normal-light images, enables better generalization than synthetic datasets, as validated by improved performance on public benchmarks.
• The frequency loss component contributes significantly to detail preservation, as removing it reduces PSNR by 0.451 and SSIM by 0.011, indicating its role in high-frequency detail recovery.
• Visual comparisons show that R2RNet effectively suppresses noise and artifacts in dark regions, while avoiding the blur and color distortion seen in RetinexNet and under-enhancement in SRIE.