[論文レビュー] Low-Light Image Enhancement with Normalizing Flow

Yufei Wang, Renjie Wan+4 more|arXiv (Cornell University)|Sep 13, 2021

Image Enhancement Techniques被引用数 33

ひとこと要約

LLFlow は、条件付き正規化フローを用いて、通常露出画像の条件付き分布をモデル化し、低照度入力からの多様性のある強化を可能にし、明るさの改善、ノイズ/アーティファクトの低減、カラーの豊かさを生み出す。

ABSTRACT

To enhance low-light images to normally-exposed ones is highly ill-posed, namely that the mapping relationship between them is one-to-many. Previous works based on the pixel-wise reconstruction losses and deterministic processes fail to capture the complex conditional distribution of normally exposed images, which results in improper brightness, residual noise, and artifacts. In this paper, we investigate to model this one-to-many relationship via a proposed normalizing flow model. An invertible network that takes the low-light images/features as the condition and learns to map the distribution of normally exposed images into a Gaussian distribution. In this way, the conditional distribution of the normally exposed images can be well modeled, and the enhancement process, i.e., the other inference direction of the invertible network, is equivalent to being constrained by a loss function that better describes the manifold structure of natural images during the training. The experimental results on the existing benchmark datasets show our method achieves better quantitative and qualitative results, obtaining better-exposed illumination, less noise and artifact, and richer colors.

研究の動機と目的

• Motivate the need to move beyond pixel-wise losses due to one-to-many mappings in low-light enhancement.
• Propose a conditional normalizing flow framework to learn the distribution of well-exposed images conditioned on a low-light input.
• Incorporate an illumination-invariant color map via a Retinex-inspired encoder to capture global image properties.
• Demonstrate improved quantitative and qualitative results on public benchmarks compared to state-of-the-art methods.
• Provide ablations to justify each component and training strategy within LLFlow.

提案手法

• Use a conditional normalizing flow to map a low-light image to a distribution of normally exposed images via a latent code z with f_flow(x|x_l).
• Encode an illumination-invariant color map g(x_l) from the low-light input to condition the flow and capture global scene properties.
• Incorporate a Retinex-inspired color map C(x) and a noise map N(x) as inputs to the encoder to improve color fidelity and robustness to noise.
• Train by maximizing the exact likelihood with the change-of-variables formula, using an encoder-conditioned latent prior f_z(z) and a random selector r(a,b) for mean conditioning.
• During inference, sample z from N(g(x_l), I) or use z = g(x_l) for faster results to generate normally exposed images.
• Adopt an ablation-friendly design to compare NLL training against L1 and demonstrate the benefits of the flow-based approach.

実験結果

リサーチクエスチョン

• RQ1 Can a conditional normalizing flow better capture the multi-modal distribution of well-exposed images given a low-light input compared to pixel-wise losses?
• RQ2 Does incorporating an illumination-invariant color map as prior improve color saturation and reduce artifacts in enhanced images?
• RQ3 How does LLFlow perform on standard low-light benchmarks (LOL) and cross-dataset scenarios (VE-LOL) compared to state-of-the-art methods?
• RQ4 What is the impact of training with NLL loss versus traditional L1 loss on perceptual quality and artifacts?

主な発見

Method	PSNR	SSIM	LPIPS
LLFlow (Ours)	25.19	0.93	0.11
KinD++ (Zhang et al. 2021)	21.30	0.82	0.16
KinD (Zhang et al. 2019)	20.87	0.80	0.17
Lv, Li, and Lu (2021)	20.24	0.79	0.14
DRBN (Yang et al. 2020)	20.13	0.83	0.16
EnlightenGAN (Jiang et al. 2021)	17.48	0.65	0.32
Zero-DCE (Guo et al. 2020)	14.86	0.54	0.33

• LLFlow achieves superior PSNR, SSIM, and LPIPS on LOL, outperforming competitors (e.g., LLFlow PSNR 25.19, SSIM 0.93, LPIPS 0.11).
• Cross-dataset evaluation shows LLFlow attaining the best quantitative results on VE-LOL when trained on LOL (PSNR 23.85, SSIM 0.8986, LPIPS 0.1456).
• Intra-dataset VE-LOL results (re-trained on VE-LOL) show LLFlow leading with PSNR 26.02, SSIM 0.9266, LPIPS 0.0996.
• Ablations indicate that NLL training significantly outperforms L1-based training and that the illumination-invariant color map improves alignment to reference images.
• Visual analyses (gradient activation maps) demonstrate LLFlow’s ability to localize artifact-prone regions and constrain unrealistic areas.
• Sampling experiments show brightness varies monotonically with the latent variable z, indicating better encoding of brightness variance.