[Paper Review] End-to-end Interpretable Learning of Non-blind Image Deblurring

Thomas Eboli, Jian Sun+1 more|arXiv (Cornell University)|Jul 3, 2020

Advanced Image Processing Techniques45 references4 citations

TL;DR

This paper proposes a learnable, end-to-end interpretable framework for non-blind image deblurring using a convolutional preconditioned Richardson fixed-point solver. By embedding approximate inverse filters of blur and prior kernels into a CNN-based architecture, it achieves state-of-the-art performance in both uniform and non-uniform deblurring, significantly outperforming FFT and conjugate gradient methods in speed and accuracy while remaining robust to approximate blur kernels.

ABSTRACT

Non-blind image deblurring is typically formulated as a linear least-squares problem regularized by natural priors on the corresponding sharp picture's gradients, which can be solved, for example, using a half-quadratic splitting method with Richardson fixed-point iterations for its least-squares updates and a proximal operator for the auxiliary variable updates. We propose to precondition the Richardson solver using approximate inverse filters of the (known) blur and natural image prior kernels. Using convolutions instead of a generic linear preconditioner allows extremely efficient parameter sharing across the image, and leads to significant gains in accuracy and/or speed compared to classical FFT and conjugate-gradient methods. More importantly, the proposed architecture is easily adapted to learning both the preconditioner and the proximal operator using CNN embeddings. This yields a simple and efficient algorithm for non-blind image deblurring which is fully interpretable, can be learned end to end, and whose accuracy matches or exceeds the state of the art, quite significantly, in the non-uniform case.

Motivation & Objective

• To address the limitations of classical solvers like FFT and conjugate gradient in non-blind image deblurring, especially in non-uniform blur scenarios.
• To develop a fast and accurate least-squares solver that exploits convolutional structure for efficient parameter sharing across the image.
• To enable end-to-end learning of both the preconditioner and proximal operator using CNNs, improving generalization and performance.
• To ensure interpretability and robustness by using physically grounded preconditioners instead of black-box neural networks.
• To extend the method to non-uniform motion blur by precomputing approximate inverses for discrete local blur kernels.

Proposed method

• Uses a half-quadratic splitting framework to decouple the deblurring problem into a least-squares subproblem and a proximal subproblem.
• Replaces standard least-squares solvers with a preconditioned Richardson fixed-point iteration using convolutional filters as preconditioners.
• Employs approximate inverse filters of the blur kernel and natural image prior kernel as learnable convolutional layers in a CNN.
• Integrates the preconditioner and proximal operator into a single end-to-end trainable network, enabling joint optimization.
• Precomputes approximate inverse filters for a discrete set of local blur kernels (e.g., translations in {1,3,...,35} and orientations in {0°,6°,...,174°}) to enable efficient non-uniform deblurring.
• Operates directly in the pixel domain, avoiding Fourier domain limitations such as boundary artifacts and periodicity assumptions.

Experimental results

Research questions

• RQ1 Can a convolutional preconditioner outperform FFT and conjugate gradient methods in solving the least-squares subproblem of non-blind deblurring?
• RQ2 Can the preconditioner and proximal operator be jointly learned in an end-to-end fashion while preserving interpretability?
• RQ3 How does the method perform on non-uniform motion blur, where FFT-based methods are inapplicable?
• RQ4 Can the method remain robust when using approximate or large blur kernels not seen during training?
• RQ5 Does the proposed approach achieve state-of-the-art performance in both uniform and non-uniform deblurring settings?

Key findings

• The proposed method, LCHQS, achieves a +1.13 dB PSNR gain over CG-based methods on non-uniform deblurring with 1% noise, while being 200× faster.
• For non-uniform deblurring with 1% noise, LCHQS_F achieves 26.98 ± 0.08 dB PSNR, outperforming HQS-CG (25.84 dB) and HQS-FFT (23.49 dB).
• The method is robust to approximate blur kernels: LCHQS_F deblurs real-world images with a 101×101 estimated kernel, producing sharper results than FCNN and FDN without introducing artifacts.
• In the non-blind setting, LCHQS_F achieves 26.98 ± 0.08 dB PSNR on 1% noise, significantly outperforming CG-based and FFT-based baselines.
• The method is 200× faster than CG and 420× faster than FFT-based solvers on 500×375 RGB images, with inference times under 1 second.
• The approach generalizes well to unseen blur kernels and noise levels, demonstrating robustness without fine-tuning.