[Paper Review] A Dynamic Kernel Prior Model for Unsupervised Blind Image Super-Resolution

Zhixiong Yang, Jingyuan Xia+6 more|arXiv (Cornell University)|Apr 24, 2024

Advanced Image Processing Techniques4 citations

TL;DR

This paper proposes a dynamic kernel prior (DKP) model for unsupervised blind image super-resolution, enabling pre-training-free, plug-and-play kernel estimation via Markov Chain Monte Carlo (MCMC) sampling and network-based Langevin dynamics. The DKP achieves state-of-the-art performance in both Gaussian and motion blur scenarios by learning adaptive kernel priors, outperforming existing methods while maintaining comparable runtime and memory usage.

ABSTRACT

Deep learning-based methods have achieved significant successes on solving the blind super-resolution (BSR) problem. However, most of them request supervised pre-training on labelled datasets. This paper proposes an unsupervised kernel estimation model, named dynamic kernel prior (DKP), to realize an unsupervised and pre-training-free learning-based algorithm for solving the BSR problem. DKP can adaptively learn dynamic kernel priors to realize real-time kernel estimation, and thereby enables superior HR image restoration performances. This is achieved by a Markov chain Monte Carlo sampling process on random kernel distributions. The learned kernel prior is then assigned to optimize a blur kernel estimation network, which entails a network-based Langevin dynamic optimization strategy. These two techniques ensure the accuracy of the kernel estimation. DKP can be easily used to replace the kernel estimation models in the existing methods, such as Double-DIP and FKP-DIP, or be added to the off-the-shelf image restoration model, such as diffusion model. In this paper, we incorporate our DKP model with DIP and diffusion model, referring to DIP-DKP and Diff-DKP, for validations. Extensive simulations on Gaussian and motion kernel scenarios demonstrate that the proposed DKP model can significantly improve the kernel estimation with comparable runtime and memory usage, leading to state-of-the-art BSR results. The code is available at https://github.com/XYLGroup/DKP.

Motivation & Objective

• To address the limitations of supervised pre-training in blind super-resolution (BSR) methods, which require expensive paired LR-HR datasets.
• To develop an unsupervised, plug-and-play kernel estimation framework that avoids training on labeled data.
• To improve kernel estimation accuracy in ill-posed, non-convex BSR problems using adaptive, data-driven priors.
• To enable compatibility with existing image restoration models such as DIP and diffusion models without architectural changes.
• To achieve state-of-the-art performance in both Gaussian and motion blur scenarios without relying on ground-truth HR images.

Proposed method

• The DKP model consists of two modules: Random Kernel Sampling (RKS) and Prior Kernel Estimation (PKE).
• The RKS module uses MCMC sampling on random kernel distributions to generate rational, adaptive kernel priors in an unsupervised manner.
• The PKE module employs a network-based Langevin dynamics (NLD) optimization strategy to estimate blur kernels using the RKS-generated prior and data consistency from LR reconstruction error.
• The NLD update scheme ensures convergence to accurate kernel estimates by balancing prior knowledge and observed data.
• DKP is integrated into existing restoration models—DIP and diffusion models—via plug-and-play integration, replacing or augmenting their kernel modeling components.
• The entire system is trained end-to-end in an alternating optimization scheme, jointly refining HR image reconstruction and kernel estimation without pre-training.

Experimental results

Research questions

• RQ1 Can an unsupervised kernel prior be effectively learned without relying on paired LR-HR training data?
• RQ2 Can MCMC sampling on random kernel distributions generate rational, adaptive priors that improve kernel estimation in blind super-resolution?
• RQ3 Does the integration of a dynamic kernel prior via NLD optimization enhance image restoration performance in both Gaussian and motion blur scenarios?
• RQ4 How does the DKP model compare in performance, runtime, and memory usage to existing pre-trained or supervised BSR methods?
• RQ5 Can DKP be seamlessly plugged into off-the-shelf restoration models like DIP and diffusion models without architectural modifications?

Key findings

• The DKP model achieves a kernel PSNR of 56.76 in the motion blur scenario and 56.20 in the Gaussian blur scenario, significantly outperforming existing unsupervised methods.
• In the DIP-DKP configuration, the method achieves an image PSNR of 25.98 and SSIM of 0.6878, surpassing Double-DIP and DIP-FKP in both metrics.
• The Diff-DKP variant achieves a state-of-the-art image PSNR of 26.76 and SSIM of 0.7400 on motion blur data, demonstrating superior restoration quality.
• Abllation studies confirm that removing the RKS module causes a dramatic drop in kernel estimation accuracy (e.g., kernel PSNR drops from 56.76 to 34.78), proving its critical role.
• The DKP model maintains moderate computational costs: 92 seconds and 11 GB memory on RTX 3090 for 512×512 image restoration, comparable to Double-DIP and DIP-FKP.
• The PKE module shows robust generalization across different network depths (3–7 layers) and width (100–1000 units), indicating low sensitivity to architecture design.