[Paper Review] Kernel-Free Image Deblurring with a Pair of Blurred/Noisy Images.

Chunzhi Gu, Xuequan Lu+2 more|arXiv (Cornell University)|Mar 26, 2019

Advanced Image Processing Techniques57 references4 citations

TL;DR

This paper proposes a kernel-free image deblurring method that leverages a pair of blurred (low shutter speed, low ISO) and noisy (high shutter speed, high ISO) images captured in low-light conditions. By modeling patch-wise intensity distributions via a GMM using optical flow-aligned patches and refining with a bilateral term and detail layer, the method achieves state-of-the-art performance in visual quality and quantitative metrics without blur kernel estimation.

ABSTRACT

Complex blur like the mixup of space-variant and space-invariant blur, which is hard to be modeled mathematically, widely exists in real images. In the real world, a common type of blur occurs when capturing images in low-light environments. In this paper, we propose a novel image deblurring method that does not need to estimate blur kernels. We utilize a pair of images which can be easily acquired in low-light situations: (1) a blurred image taken with low shutter speed and low ISO noise, and (2) a noisy image captured with high shutter speed and high ISO noise. Specifically, the blurred image is first sliced into patches, and we extend the Gaussian mixture model (GMM) to model the underlying intensity distribution of each patch using the corresponding patches in the noisy image. We compute patch correspondences by analyzing the optical flow between the two images. The Expectation-Maximization (EM) algorithm is utilized to estimate the involved parameters in the GMM. To preserve sharp features, we add an additional bilateral term to the objective function in the M-step. We eventually add a detail layer to the deblurred image for refinement. Extensive experiments on both synthetic and real-world data demonstrate that our method outperforms state-of-the-art techniques, in terms of robustness, visual quality and quantitative metrics. We will make our dataset and source code publicly available.

Motivation & Objective

• To address complex, real-world blur types—especially space-variant and space-invariant mixups—that are difficult to model mathematically.
• To eliminate the need for blur kernel estimation, which is a major limitation in existing deblurring methods.
• To leverage easily obtainable image pairs from low-light photography: one blurred (low shutter speed, low noise) and one noisy (high shutter speed, high noise).
• To improve deblurring robustness and visual quality in challenging real-world scenarios.

Proposed method

• The blurred image is divided into patches, and each patch's intensity distribution is modeled using an extended Gaussian Mixture Model (GMM) trained on corresponding patches from the noisy image.
• Patch correspondences are established via optical flow estimation between the blurred and noisy images to align spatially corresponding regions.
• The Expectation-Maximization (EM) algorithm is applied to estimate GMM parameters, incorporating a bilateral term in the M-step to preserve sharp edges and fine details.
• A detail layer is added to the final deblurred image to refine high-frequency components and enhance texture fidelity.
• The method avoids explicit blur kernel learning by relying on statistical modeling of intensity distributions across the image pair.
• The framework is trained and evaluated on both synthetic and real-world datasets, with public release of code and data.

Experimental results

Research questions

• RQ1 Can a kernel-free deblurring method achieve superior performance by leveraging a pair of blurred and noisy images captured in low-light conditions?
• RQ2 How effectively can a GMM-based patch intensity model, trained on noisy image patches, reconstruct sharpness in the deblurred output without kernel estimation?
• RQ3 To what extent does the inclusion of a bilateral term in the EM-based optimization improve edge preservation and visual quality?
• RQ4 How does the proposed method compare to state-of-the-art kernel-based and kernel-free deblurring techniques in terms of robustness and quantitative metrics?

Key findings

• The proposed method outperforms existing state-of-the-art deblurring techniques in both quantitative metrics and visual quality on synthetic and real-world datasets.
• The use of a GMM with optical flow-aligned patches enables accurate modeling of complex blur distributions without requiring explicit kernel estimation.
• The addition of a bilateral term in the M-step significantly improves edge preservation and sharpness in the deblurred output.
• The detail layer refinement step enhances high-frequency textures and fine structures, contributing to improved perceptual quality.
• The method demonstrates strong robustness to complex, real-world blur patterns such as space-variant and mixed blur types.
• The authors confirm the method's effectiveness through extensive ablation studies and public release of dataset and code.