[Paper Review] Deep Richardson-Lucy Deconvolution for Low-Light Image Deblurring

Liang Chen, Jiawei Zhang+5 more|arXiv (Cornell University)|Aug 10, 2023

Advanced Image Processing Techniques4 citations

TL;DR

This paper proposes a data-driven deep learning method for low-light image deblurring that models saturated pixels via a learned latent map, integrating a prior estimation network (PEN) with a Richardson-Lucy (RL)-based optimization scheme. The approach outperforms state-of-the-art methods on both synthetic and real-world datasets by effectively handling blur and saturation without relying on handcrafted clipping approximations.

ABSTRACT

Images taken under the low-light condition often contain blur and saturated pixels at the same time. Deblurring images with saturated pixels is quite challenging. Because of the limited dynamic range, the saturated pixels are usually clipped in the imaging process and thus cannot be modeled by the linear blur model. Previous methods use manually designed smooth functions to approximate the clipping procedure. Their deblurring processes often require empirically defined parameters, which may not be the optimal choices for different images. In this paper, we develop a data-driven approach to model the saturated pixels by a learned latent map. Based on the new model, the non-blind deblurring task can be formulated into a maximum a posterior (MAP) problem, which can be effectively solved by iteratively computing the latent map and the latent image. Specifically, the latent map is computed by learning from a map estimation network (MEN), and the latent image estimation process is implemented by a Richardson-Lucy (RL)-based updating scheme. To estimate high-quality deblurred images without amplified artifacts, we develop a prior estimation network (PEN) to obtain prior information, which is further integrated into the RL scheme. Experimental results demonstrate that the proposed method performs favorably against state-of-the-art algorithms both quantitatively and qualitatively on synthetic and real-world images.

Motivation & Objective

• To address the challenge of deblurring low-light images that contain both blur and saturated pixels.
• To overcome the limitations of prior methods that rely on manually designed smooth functions to approximate pixel clipping.
• To eliminate the need for empirically tuned parameters in deblurring by learning a data-driven representation of saturated pixels.
• To improve deblurring quality by integrating prior information through a learned prior estimation network (PEN).
• To formulate the non-blind deblurring task as a maximum a posteriori (MAP) problem solvable via iterative latent map and image updates.

Proposed method

• A map estimation network (MEN) is trained to predict a latent map representing saturated pixels, replacing traditional handcrafted clipping models.
• The latent image is updated iteratively using a Richardson-Lucy (RL)-based scheme, which is well-suited for Poisson noise and blur modeling.
• A prior estimation network (PEN) learns spatial priors to guide the deblurring process and suppress artifacts.
• The method formulates non-blind deblurring as a MAP problem, jointly optimizing the latent image and the learned latent map.
• The iterative optimization alternates between updating the latent image via RL and refining the latent map via MEN, with PEN-provided priors enhancing stability and quality.
• The framework is trained end-to-end on synthetic low-light deblurring data, enabling generalization to real-world images.

Experimental results

Research questions

• RQ1 Can a learned latent map effectively model saturated pixels in low-light images without relying on handcrafted clipping functions?
• RQ2 How does integrating a learned prior via PEN improve deblurring performance and reduce artifacts?
• RQ3 Can the proposed data-driven RL-based framework outperform existing methods on both synthetic and real-world low-light deblurring benchmarks?
• RQ4 Does the MAP formulation with iterative latent map and image updates lead to more robust and accurate deblurring than prior optimization strategies?
• RQ5 To what extent does the method generalize to real-world low-light images with complex blur and saturation patterns?

Key findings

• The proposed method achieves state-of-the-art performance on both synthetic and real-world low-light image deblurring benchmarks, as measured by PSNR and SSIM.
• The method significantly reduces artifacts and noise amplification compared to baseline approaches, particularly in high-saturation regions.
• The integration of the prior estimation network (PEN) leads to more natural-looking deblurred results with improved structural preservation.
• The model generalizes well to real-world images, demonstrating robustness to complex blur kernels and varying lighting conditions.
• Abllation studies confirm that both the learned latent map and the PEN component are essential for optimal performance.
• The method outperforms existing non-blind deblurring methods in both quantitative metrics and qualitative visual assessment.