[论文解读] Structure-Preserving Super Resolution with Gradient Guidance

Cheng Ma, Yongming Rao+4 more|arXiv (Cornell University)|Mar 29, 2020

Advanced Image Processing Techniques参考文献 45被引用 38

一句话总结

基于GAN的单图像超分辨方法（SPSR），通过梯度分支和梯度损失在保持几何结构的同时实现强感知质量，并具备具有竞争力的 PSNR/SSIM。

ABSTRACT

Structures matter in single image super resolution (SISR). Recent studies benefiting from generative adversarial network (GAN) have promoted the development of SISR by recovering photo-realistic images. However, there are always undesired structural distortions in the recovered images. In this paper, we propose a structure-preserving super resolution method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details. Specifically, we exploit gradient maps of images to guide the recovery in two aspects. On the one hand, we restore high-resolution gradient maps by a gradient branch to provide additional structure priors for the SR process. On the other hand, we propose a gradient loss which imposes a second-order restriction on the super-resolved images. Along with the previous image-space loss functions, the gradient-space objectives help generative networks concentrate more on geometric structures. Moreover, our method is model-agnostic, which can be potentially used for off-the-shelf SR networks. Experimental results show that we achieve the best PI and LPIPS performance and meanwhile comparable PSNR and SSIM compared with state-of-the-art perceptual-driven SR methods. Visual results demonstrate our superiority in restoring structures while generating natural SR images.

研究动机与目标

• Motivate the need to reduce geometric distortions in perceptual-driven SR methods.
• Propose a gradient-guided, structure-preserving SR framework compatible with off-the-shelf SR networks.
• Improve perceptual quality (PI/LPIPS) while maintaining competitive PSNR/SSIM.
• Introduce a gradient branch that translates LR gradient maps to HR and guides SR reconstruction.
• Introduce a gradient loss that imposes second-order consistency on recovered images.

提出的方法

• Two-branch generator comprising an SR branch and a gradient branch.
• Gradient branch translates LR gradient maps to HR using features from the SR branch as priors.
• Attentive fusion: fuse gradient branch features with SR features via an RRDB-based fusion block.
• Standard image-space losses (L1, perceptual, adversarial) plus a gradient-space loss.
• Gradient loss includes a pixelwise gradient map loss (L1 on M(.) of SR vs HR) and a gradient-discriminator loss (D_GM) on gradient maps.
• Model-agnostic design enabling integration with off-the-shelf SR networks.

实验结果

研究问题

• RQ1 How can gradient information be leveraged to preserve geometric structures in GAN-based SR?
• RQ2 Does a gradient-guided SR framework reduce structural distortions while maintaining high perceptual quality and competitive PSNR/SSIM?
• RQ3 What is the impact of gradient branch and gradient loss on PV/LPIPS and structural fidelity across standard benchmarks?

主要发现

• SPSR achieves best PI and LPIPS across tested datasets while maintaining competitive PSNR and SSIM.
• In qualitative results, SPSR better preserves structures and gradients than other perceptual-driven SR methods.
• A gradient branch plus gradient loss yields sharper, more structure-preserving SR results than ESRGAN baseline.
• Ablation shows removing the gradient branch or gradient loss degrades perceptual metrics and structure preservation.
• The gradient branch recovers HR-like gradient maps from LR gradients, guiding SR to maintain geometric fidelity.