[Paper Review] Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark

Xin Lin, Jingtong Yue+4 more|arXiv (Cornell University)|May 6, 2023

Image Enhancement Techniques4 citations

TL;DR

This paper proposes $L^{2}RIRNet$, an end-to-end deep learning framework that jointly performs low-light enhancement and deraining in real-world rainy and dark conditions. By employing a Pairwise Degradation Feature Vector Extraction Network (P-Net) to separately model degradation in bright and dark regions and a Fast Fourier-ResNet Detail Guidance (FFR-DG) module for texture prior, the method achieves state-of-the-art performance on both synthetic and real-world datasets, outperforming cascaded and single-task approaches in PSNR, SSIM, and NIQE metrics.

ABSTRACT

Rain in the dark poses a significant challenge to deploying real-world applications such as autonomous driving, surveillance systems, and night photography. Existing low-light enhancement or deraining methods struggle to brighten low-light conditions and remove rain simultaneously. Additionally, cascade approaches like ``deraining followed by low-light enhancement'' or the reverse often result in problematic rain patterns or overly blurred and overexposed images. To address these challenges, we introduce an end-to-end model called L$^{2}$RIRNet, designed to manage both low-light enhancement and deraining in real-world settings. Our model features two main components: a Dual Degradation Representation Network (DDR-Net) and a Restoration Network. The DDR-Net independently learns degradation representations for luminance effects in dark areas and rain patterns in light areas, employing dual degradation loss to guide the training process. The Restoration Network restores the degraded image using a Fourier Detail Guidance (FDG) module, which leverages near-rainless detailed images, focusing on texture details in frequency and spatial domains to inform the restoration process. Furthermore, we contribute a dataset containing both synthetic and real-world low-light-rainy images. Extensive experiments demonstrate that our L$^{2}$RIRNet performs favorably against existing methods in both synthetic and complex real-world scenarios. All the code and dataset can be found in \url{https://github.com/linxin0/Low_light_rainy}.

Motivation & Objective

• Address the challenge of simultaneously restoring low-light-rainy images, which existing single-task or cascaded methods fail to handle effectively.
• Overcome limitations of cascade approaches that cause overexposure, underexposure, residual rain patterns, or blurring in low-light conditions.
• Develop a unified framework capable of handling complex real-world degradation patterns in both dark and bright regions.
• Introduce a novel degradation representation that captures distinct degradation characteristics in low-light and well-lit areas using contrastive learning.
• Enhance image restoration by integrating texture and frequency-domain prior knowledge through a new detail guidance module.

Proposed method

• Proposes $L^{2}RIRNet$, an end-to-end U-Net-based architecture composed of a Pairwise Degradation Feature Vector Extraction Network (P-Net) and a Restoration Network (R-Net).
• P-Net uses shared encoders to extract degradation feature vectors separately for dark and bright regions, leveraging contrastive learning to improve guidance accuracy.
• Introduces the Fast Fourier-ResNet Detail Guidance (FFR-DG) module that generates a detail image via channel-wise subtraction and concatenation, preserving texture while suppressing rain and lighting artifacts.
• The FFR-DG module fuses spatial and frequency-domain features using Fourier transformation to provide robust prior information for the R-Net.
• Employs a dual-branch feature learning strategy in P-Net to model heterogeneous degradation distributions across the image, improving restoration fidelity.
• Trains the model on a newly constructed dataset combining synthetic and real-world low-light-rainy image pairs to improve generalization.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework effectively address both low-light enhancement and deraining simultaneously, outperforming cascaded or single-task methods?
• RQ2 How does modeling degradation features separately in dark and bright regions improve restoration performance compared to a single unified representation?
• RQ3 To what extent does incorporating frequency-domain and spatial detail priors enhance image restoration in low-light-rainy conditions?
• RQ4 Does the proposed FFR-DG module significantly improve texture recovery and reduce artifacts like blurring or residual rain streaks?
• RQ5 How does the model generalize across synthetic and real-world low-light-rainy image distributions?

Key findings

• $L^{2}RIRNet$ achieves the highest PSNR (29.96 dB) and SSIM on the synthetic LLR dataset, significantly outperforming all baseline and cascaded methods.
• On the real-world LLR dataset, $L^{2}RIRNet$ achieves the lowest NIQE score of 13.881, indicating superior perceptual quality compared to other methods.
• Ablation studies show that adding the P-Net improves PSNR by 0.58 dB over single-branch degradation feature extraction, proving the value of dual-region modeling.
• The inclusion of the FFR-DG module increases PSNR by 0.84 dB compared to the P-Net-only variant, demonstrating its critical role in detail recovery.
• Even after retraining state-of-the-art models (e.g., MIRNet, RCDNet) on the LLR dataset, $L^{2}RIRNet$ maintains superior performance in both quantitative and qualitative evaluations.
• Visual results confirm that $L^{2}RIRNet$ effectively removes rain streaks without overexposure or blurring, while restoring visibility in dark regions—unlike cascaded methods that leave residual artifacts.