[Paper Review] Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation

Dong Gong, Wei Sun+3 more|arXiv (Cornell University)|Jan 8, 2020

Advanced Image Processing Techniques32 references4 citations

TL;DR

This paper proposes a novel unpaired real-world super-resolution method that bypasses the need for aligned LR-HR image pairs by training a degradation generation network to synthesize realistic low-resolution images from high-resolution ones, then minimizing domain gap via feature-level alignment during super-resolution training. The approach achieves state-of-the-art performance on real-world datasets, even outperforming methods trained on paired data.

ABSTRACT

Most learning-based super-resolution (SR) methods aim to recover high-resolution (HR) image from a given low-resolution (LR) image via learning on LR-HR image pairs. The SR methods learned on synthetic data do not perform well in real-world, due to the domain gap between the artificially synthesized and real LR images. Some efforts are thus taken to capture real-world image pairs. The captured LR-HR image pairs usually suffer from unavoidable misalignment, which hampers the performance of end-to-end learning, however. Here, focusing on the real-world SR, we ask a different question: since misalignment is unavoidable, can we propose a method that does not need LR-HR image pairing and alignment at all and utilize real images as they are? Hence we propose a framework to learn SR from an arbitrary set of unpaired LR and HR images and see how far a step can go in such a realistic and "unsupervised" setting. To do so, we firstly train a degradation generation network to generate realistic LR images and, more importantly, to capture their distribution (i.e., learning to zoom out). Instead of assuming the domain gap has been eliminated, we minimize the discrepancy between the generated data and real data while learning a degradation adaptive SR network (i.e., learning to zoom in). The proposed unpaired method achieves state-of-the-art SR results on real-world images, even in the datasets that favor the paired-learning methods more.

Motivation & Objective

• To address the domain gap between synthetic and real-world low-resolution images in super-resolution.
• To eliminate the need for paired, aligned LR-HR image data in real-world super-resolution.
• To develop a method that learns to super-resolve real images using only unpaired LR and HR image sets.
• To minimize the discrepancy between generated and real LR images through adversarial and cycle-consistency training.
• To improve robustness and generalization by aligning network responses across real and generated data distributions.

Proposed method

• A cycle-GAN-based degradation generation network is trained to synthesize realistic LR images from HR images, mimicking real-world degradation processes.
• The network uses adversarial loss and cycle-consistency loss to learn the distribution of real LR images without requiring paired supervision.
• A degradation-adaptive super-resolution network is trained by minimizing feature-level domain discrepancy between real and generated LR images.
• Feature-level alignment is achieved by aligning intermediate and output features of the SR network across real and generated LR inputs.
• The method operates in an unsupervised, unpaired setting, avoiding reliance on image pairing or pre-registration.
• The framework is evaluated under both standard and non-overlapping learning settings to test robustness.

Experimental results

Research questions

• RQ1 Can real-world super-resolution be effectively learned without paired or aligned LR-HR image data?
• RQ2 Can a degradation generation network accurately model the complex distribution of real-world LR images?
• RQ3 Does minimizing domain gap at the feature level improve super-resolution performance on real-world images?
• RQ4 How does the proposed unpaired method compare to paired and misaligned training baselines on real-world datasets?
• RQ5 Can the method generalize across diverse real-world degradation patterns without overlapping content between LR and HR sets?

Key findings

• The proposed method achieves state-of-the-art performance on real-world super-resolution benchmarks, outperforming models trained on paired data such as RCAN-Real and ESRGAN-Bic.
• Visual comparisons show that the method produces sharper textures and fewer artifacts, especially in complex regions like text and fine patterns.
• The degradation generation network successfully captures realistic degradation characteristics, producing LR images that closely resemble real-world LR images in distribution.
• Even under a non-overlapping learning setting—where LR and HR images have no common content—the method maintains strong performance, demonstrating robustness.
• Quantitative results on RealSR, SR-RGB, and City100 datasets confirm consistent gains in PSNR and SSIM over prior unpaired methods like CinCGAN.
• The method reduces blur and structural artifacts compared to baseline models trained on synthetic bicubic degradation, indicating better generalization to real-world data.