[Paper Review] DIVA: Deep Unfolded Network from Quantum Interactive Patches for Image Restoration

Sayantan Dutta, Adrian Basarab+2 more|arXiv (Cornell University)|Dec 31, 2022

Image and Signal Denoising Methods70 references4 citations

TL;DR

This paper proposes DIVA, a deep unfolded neural network that leverages quantum many-body physics to enhance image restoration, using a Hamiltonian-based interaction mechanism to model patch-level relationships. By unfolding the De-QuIP algorithm and enabling patch-wise hyperparameter learning via back-propagation, DIVA achieves state-of-the-art visual quality and competitive PSNR/SSIM scores across denoising, deblurring, super-resolution, and inpainting tasks with minimal architectural complexity.

ABSTRACT

This paper presents a deep neural network called DIVA unfolding a baseline adaptive denoising algorithm (De-QuIP), relying on the theory of quantum many-body physics. Furthermore, it is shown that with very slight modifications, this network can be enhanced to solve more challenging image restoration tasks such as image deblurring, super-resolution and inpainting. Despite a compact and interpretable (from a physical perspective) architecture, the proposed deep learning network outperforms several recent algorithms from the literature, designed specifically for each task. The key ingredients of the proposed method are on one hand, its ability to handle non-local image structures through the patch-interaction term and the quantum-based Hamiltonian operator, and, on the other hand, its flexibility to adapt the hyperparameters patch-wisely, due to the training process.

Motivation & Objective

• To develop a unified deep learning framework for multiple image restoration tasks using principles from quantum many-body theory.
• To improve the interpretability and adaptability of deep networks by embedding physical principles from quantum mechanics into the architecture.
• To enable end-to-end trainable, patch-wise hyperparameter tuning for enhanced performance across diverse degradation models.
• To extend a single-purpose denoising algorithm (De-QuIP) into a versatile restoration framework applicable to deblurring, super-resolution, and inpainting.
• To reduce training cost and improve generalization by leveraging quantum-inspired interactions without task-specific architectural redesign.

Proposed method

• The method unfolds the De-QuIP algorithm—a quantum many-body physics-based denoising approach—into a deep convolutional neural network architecture.
• It introduces a quantum-inspired Hamiltonian operator that models non-local interactions between image patches, capturing structural similarities.
• The network employs a patch-wise interaction layer that dynamically adjusts parameters such as the proportionality constant and thresholding energy via back-propagation.
• The architecture is trained end-to-end using clean-degraded image pairs, enabling adaptive tuning of physical hyperparameters per patch.
• The framework integrates the Schrödinger equation’s formalism into a learnable loss function, linking quantum dynamics to image reconstruction.
• The model is flexible and can be adapted to multiple tasks (denoising, deblurring, super-resolution, inpainting) with minimal architectural changes.

Experimental results

Research questions

• RQ1 Can a quantum many-body physics-inspired algorithm be effectively unfolded into a deep learning framework for image restoration?
• RQ2 How does patch-wise adaptive learning of physical parameters improve performance across diverse image restoration tasks?
• RQ3 To what extent does the quantum interaction mechanism enhance non-local structure recovery compared to conventional CNNs?
• RQ4 Can a unified network architecture achieve competitive results across multiple image restoration tasks without task-specific design?
• RQ5 How does the proposed method compare quantitatively and qualitatively to SOTA methods in challenging degradation scenarios?

Key findings

• DIVA achieves state-of-the-art visual quality in image restoration, particularly excelling in texture and structural preservation, even when quantitative metrics are slightly lower than some SOTA models.
• For image inpainting with 50% missing pixels, DIVA achieves 36.08 dB PSNR and 97.84% SSIM on Set5, outperforming IRCNN (35.57 dB/95.87% SSIM).
• On the Set12 dataset with 80% missing pixels, DIVA achieves 27.73 dB PSNR and 91.92% SSIM, significantly outperforming IRCNN (26.75 dB/84.53% SSIM).
• In super-resolution and deblurring tasks, DIVA demonstrates competitive PSNR and SSIM values, with superior visual fidelity and detail preservation compared to benchmarks.
• The model shows robustness in handling complex degradation patterns, particularly excelling in preserving image textures and patterns through its quantum-inspired interaction mechanism.
• Despite lower quantitative scores in Gaussian downsampling tasks (e.g., compared to DRLN or RDN), DIVA maintains strong visual quality, indicating a trade-off between metric scores and perceptual performance.