[Paper Review] Attention Guided Low-light Image Enhancement with a Large Scale Low-light Simulation Dataset

Feifan Lv, Yu Li+1 more|arXiv (Cornell University)|Aug 2, 2019

Image Enhancement Techniques81 references47 citations

TL;DR

The paper proposes an end-to-end, attention-guided multi-branch network for simultaneous low-light enhancement and denoising, trained on a large-scale synthetic low-light dataset.

ABSTRACT

Low-light image enhancement is challenging in that it needs to consider not only brightness recovery but also complex issues like color distortion and noise, which usually hide in the dark. Simply adjusting the brightness of a low-light image will inevitably amplify those artifacts. To address this difficult problem, this paper proposes a novel end-to-end attention-guided method based on multi-branch convolutional neural network. To this end, we first construct a synthetic dataset with carefully designed low-light simulation strategies. The dataset is much larger and more diverse than existing ones. With the new dataset for training, our method learns two attention maps to guide the brightness enhancement and denoising tasks respectively. The first attention map distinguishes underexposed regions from well lit regions, and the second attention map distinguishes noises from real textures. With their guidance, the proposed multi-branch decomposition-and-fusion enhancement network works in an input adaptive way. Moreover, a reinforcement-net further enhances color and contrast of the output image. Extensive experiments on multiple datasets demonstrate that our method can produce high fidelity enhancement results for low-light images and outperforms the current state-of-the-art methods by a large margin both quantitatively and visually.

Motivation & Objective

• Motivate robust low-light enhancement that recovers brightness while suppressing noise and color distortion.
• Build a large-scale, high-fidelity synthetic low-light dataset with paired ground-truth images and annotations.
• Develop an attention-guided, multi-branch CNN architecture that jointly denoises and enhances, guided by exposure and noise maps.
• Improve color, contrast, and detail reconstruction via a secondary reinforcement step.
• Demonstrate state-of-the-art performance on multiple datasets with quantitative and visual improvements.

Proposed method

• Construct a large-scale low-light simulation pipeline to synthesize realistic low-light images from normal-light images, including a Gaussian-Poisson mixed noise model.
• Propose an end-to-end network with four subnets: Attention-Net, Noise-Net, Enhancement-Net, and Reinforce-Net.
• Use an ue-attention map to identify underexposed regions and derive a noise map from exposure guided information.
• Implement a multi-branch Enhancement-Net that fuses outputs from multiple enhancement modules (EMs) guided by the attention signals.
• Apply a Reinforce-Net with dilated convolutions to improve contrast and detail in the final output.
• Train with a composite loss that includes Attention-Net, Noise-Net, Enhancment-Net, and Reinforce-Net components and their respective terms (bright, structural, perceptual, regional).

Experimental results

Research questions

• RQ1 How can underexposed regions and noise distributions be effectively estimated and used to guide simultaneous enhancement and denoising?
• RQ2 Can a large-scale synthetic low-light dataset improve generalization of enhancement models across diverse scenes and exposure levels?
• RQ3 Does a multi-branch network with attention guidance outperform single-branch or cascade denoising+enhancement approaches in low-light conditions?
• RQ4 Does a reinforcement step improve color and contrast beyond initial enhancement results?

Key findings

• The proposed method achieves high fidelity enhancement and outperforms state-of-the-art methods by a large margin on multiple datasets.
• The UE-attention map effectively guides enhancement to underexposed regions and helps derive a noise map for adaptive denoising.
• The multi-branch Enhancement-Net, guided by Attention-Net and Noise-Net outputs, enables simultaneous denoising and enhancement.
• A Reinforce-Net improves color and contrast, addressing low-contrast artifacts common in regression-based approaches.
• The synthetic dataset provides diverse exposure levels, realistic noise, and compatible ground truth for training and evaluation.