[论文解读] Meta-SR: A Magnification-Arbitrary Network for Super-Resolution

Xuecai Hu, Haoyuan Mu+4 more|arXiv (Cornell University)|Mar 3, 2019

Advanced Image Processing Techniques参考文献 35被引用 45

一句话总结

Meta-SR 引入一个单一模型，能够在任意（包括非整数）尺度因子下执行超分辨率，通过使用一个 Meta-Upscale 模块，该模块根据尺度和坐标预测每像素的滤波权重。

ABSTRACT

Recent research on super-resolution has achieved great success due to the development of deep convolutional neural networks (DCNNs). However, super-resolution of arbitrary scale factor has been ignored for a long time. Most previous researchers regard super-resolution of different scale factors as independent tasks. They train a specific model for each scale factor which is inefficient in computing, and prior work only take the super-resolution of several integer scale factors into consideration. In this work, we propose a novel method called Meta-SR to firstly solve super-resolution of arbitrary scale factor (including non-integer scale factors) with a single model. In our Meta-SR, the Meta-Upscale Module is proposed to replace the traditional upscale module. For arbitrary scale factor, the Meta-Upscale Module dynamically predicts the weights of the upscale filters by taking the scale factor as input and use these weights to generate the HR image of arbitrary size. For any low-resolution image, our Meta-SR can continuously zoom in it with arbitrary scale factor by only using a single model. We evaluated the proposed method through extensive experiments on widely used benchmark datasets on single image super-resolution. The experimental results show the superiority of our Meta-Upscale.

研究动机与目标

• Motivate practical SISR where users can zoom images at any scale, not just fixed integers.
• Develop a single model capable of arbitrary-scale SR to avoid maintaining multiple scale-specific models.
• Leverage meta-learning to predict per-scale weights for convolution filters during upsampling.

提出的方法

• Introduce Meta-Upscale Module that replaces traditional upsampling by predicting filter weights per output pixel.
• Use Location Projection to map SR pixels to LR coordinates via floor(i/r), floor(j/r).
• Predict per-pixel weights W(i,j) with a weight-prediction network φ that takes a scale-coordinate vector v_ij as input.
• Map LR features to SR pixels through a simple feature mapping Φ(FLR(i',j'), W(i,j)) = FLR(i',j') · W(i,j).
• Integrate Meta-Upscale with a Feature Learning Module (e.g., RDN) to form the Meta-SR network, trainable end-to-end for arbitrary r in (1,4].
• Train with L1 loss on DIV2K-derived data and evaluate on Set14, B100, Manga109, and DIV2K.

实验结果

研究问题

• RQ1 Can a single model learn SR for arbitrary scale factors, including non-integers, without storing separate weights per scale?
• RQ2 Does a dynamically predicted per-pixel upscale filter improve SR quality across a range of scales compared to fixed upscaling modules?
• RQ3 How does Meta-Upscale perform relative to scale-specific baselines when evaluating PSNR/SSIM on standard SR benchmarks?
• RQ4 What is the computational trade-off of the Meta-Upscale module compared to traditional upsampling modules?

主要发现

• Meta-SR achieves competitive or superior PSNR/SSIM on several benchmarks compared with scale-specific baselines.
• Replacing fixed upsampling with Meta-Upscale enables a single model to handle arbitrary scales more efficiently than training separate models for each scale.
• The Meta-Upscale module uses a lightweight weight-prediction network (two FC layers with ReLU activations) to generate convolution weights for each output pixel.
• For non-integer scales, the Location Projection enables fractional receptive behavior that adapts per pixel.
• Meta-SR's Meta-Upscale is faster than re-running multiple dedicated SR models when zooming across many scales.