[Paper Review] Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement

Risheng Liu, Long Ma+3 more|arXiv (Cornell University)|Dec 10, 2020

Image Enhancement TechniquesComputer Science27 references59 citations

TL;DR

RUAS integrates Retinex-inspired unrolled optimization with a cooperative, reference-free neural architecture search to build lightweight, high-performance low-light image enhancement networks. It demonstrates state-of-the-art results with low computational cost.

ABSTRACT

Low-light image enhancement plays very important roles in low-level vision field. Recent works have built a large variety of deep learning models to address this task. However, these approaches mostly rely on significant architecture engineering and suffer from high computational burden. In this paper, we propose a new method, named Retinex-inspired Unrolling with Architecture Search (RUAS), to construct lightweight yet effective enhancement network for low-light images in real-world scenario. Specifically, building upon Retinex rule, RUAS first establishes models to characterize the intrinsic underexposed structure of low-light images and unroll their optimization processes to construct our holistic propagation structure. Then by designing a cooperative reference-free learning strategy to discover low-light prior architectures from a compact search space, RUAS is able to obtain a top-performing image enhancement network, which is with fast speed and requires few computational resources. Extensive experiments verify the superiority of our RUAS framework against recently proposed state-of-the-art methods.

Motivation & Objective

• Motivate robust low-light image enhancement that preserves texture while reducing noise.
• Develop a principled network by unrolling Retinex-based optimization to model illumination and noise separately.
• Automatically discover compact prior architectures for illumination estimation and denoising without requiring paired data.
• Propose a cooperative bilevel learning framework to search and train lightweight, effective architectures.

Proposed method

• Formulate Retinex-based models with Illumination Estimation Module (IEM) and Noise Removal Module (NRM).
• Unroll optimization steps to build a propagation network where each iteration is parameterized by a CNN block.
• Define a compact search space via distillation cells with selectable operations (1x1/3x3 convs, residual convolutions, dilated convolutions, skip connections).
• Use differentiable NAS to obtain architecture parameters for IEM and NRM in a cooperative, bilevel learning setup.
• Employ reference-free losses for training: fidelity plus regularization terms (RTV for IEM, TV for NRM).
• Solve a cooperative min-min optimization over architecture parameters to jointly optimize IEM and NRM architectures.

Experimental results

Research questions

• RQ1 Can Retinex-inspired unrolling yield a lightweight yet effective hierarchy for low-light enhancement?
• RQ2 Can a cooperative, reference-free NAS strategy identify compact architectures for illumination estimation and noise removal that outperform manually designed networks?
• RQ3 How does RUAS balance brightness, texture preservation, and noise suppression across real-world low-light scenarios?
• RQ4 What are the runtime and memory benefits of the RUAS-searched architectures compared to state-of-the-art CNNs?

Key findings

• RUAS achieves state-of-the-art or competitive PSNR/SSIM on MIT-Adobe 5K and LOL datasets with significantly smaller model sizes and FLOPs.
• The cooperative search yields architectures for IEM and NRM that outperform separate or naive joint searches in PSNR, SSIM, and efficiency.
• Including the Noise Removal Module (NRM) improves performance in noisy, real-world low-light scenes.
• Searched RUAS variants (RUAS i and RUAS i+n) show superior efficiency (smaller size, lower FLOPs, faster inference) compared to several manually designed baselines.
• Ablation studies confirm the benefits of refined warm-start strategies and the necessity of NRM for noisy scenarios.