[Paper Review] FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform

Runyuan Cai, Yue Ding+1 more|arXiv (Cornell University)|Nov 21, 2021

Advanced Image Processing Techniques32 references4 citations

TL;DR

FreqNet proposes a frequency-domain super-resolution network that explicitly learns high-frequency details using discrete cosine transform (DCT) feature maps, combining spatial and frequency-domain branches with a novel frequency loss function. It achieves superior perceptual quality and high-fidelity detail reconstruction, outperforming spatial-domain methods in both metrics and visual fidelity, and enables post-processing enhancement of other SR models by merging frequency-domain outputs.

ABSTRACT

Single image super-resolution(SISR) is an ill-posed problem that aims to obtain high-resolution (HR) output from low-resolution (LR) input, during which extra high-frequency information is supposed to be added to improve the perceptual quality. Existing SISR works mainly operate in the spatial domain by minimizing the mean squared reconstruction error. Despite the high peak signal-to-noise ratios(PSNR) results, it is difficult to determine whether the model correctly adds desired high-frequency details. Some residual-based structures are proposed to guide the model to focus on high-frequency features implicitly. However, how to verify the fidelity of those artificial details remains a problem since the interpretation from spatial-domain metrics is limited. In this paper, we propose FreqNet, an intuitive pipeline from the frequency domain perspective, to solve this problem. Inspired by existing frequency-domain works, we convert images into discrete cosine transform (DCT) blocks, then reform them to obtain the DCT feature maps, which serve as the input and target of our model. A specialized pipeline is designed, and we further propose a frequency loss function to fit the nature of our frequency-domain task. Our SISR method in the frequency domain can learn the high-frequency information explicitly, provide fidelity and good perceptual quality for the SR images. We further observe that our model can be merged with other spatial super-resolution models to enhance the quality of their original SR output.

Motivation & Objective

• To address the limitation of spatial-domain SISR methods in explicitly learning and verifying high-frequency details.
• To improve perceptual quality and fidelity of super-resolved images by operating in the frequency domain.
• To develop a loss function and metric tailored for frequency-domain reconstruction of high-frequency components.
• To enable post-processing enhancement of existing SR models by merging frequency-domain outputs.
• To design domain-specific residual blocks (DWRG and DRG) that better exploit the structure of DCT feature maps.

Proposed method

• Convert LR and HR images into DCT blocks and reshape them into DCT feature maps for frequency-domain processing.
• Design a dual-branch network: a Spatial Extraction Network (SEN) for spatial features and a Frequency Reconstruction Network (FRN) for frequency-domain reconstruction.
• Implement depth-wise residual groups (DWRG) in FRN and deformable residual groups (DRG) in SEN to better capture frequency-domain and spatial characteristics.
• Propose a frequency-domain loss function $ L_{\text{freq}} $ that measures reconstruction error in the DCT domain, emphasizing high-frequency channels.
• Use a weighted sum of SEN and FRN outputs to generate final frequency-domain predictions, converted back via inverse DCT.
• Enable model fusion by replacing specific high-frequency DCT channels in other SR models’ outputs with FreqNet’s output, enhancing detail fidelity.

Experimental results

Research questions

• RQ1 Can explicit frequency-domain learning improve high-frequency detail reconstruction in single image super-resolution compared to spatial-domain methods?
• RQ2 How can a frequency-domain loss function be designed to better evaluate high-frequency detail fidelity than traditional PSNR or MSE?
• RQ3 What architectural modifications are effective for frequency-domain feature extraction, and how do they improve performance?
• RQ4 Can FreqNet’s output be effectively merged with other SR models to enhance their high-frequency detail quality?
• RQ5 To what extent does the proposed frequency-domain metric (FRM) correlate with perceptual quality and high-frequency accuracy?

Key findings

• The $ L_{\text{freq}} $-supervised FreqNet model achieved higher PSNR and FRM than the MSE-supervised baseline on Set5, with visibly sharper high-frequency textures.
• The FRM metric demonstrated improved sensitivity to high-frequency detail quality compared to PSNR, validating its effectiveness as a frequency-domain evaluation measure.
• Ablation studies showed that replacing standard residual groups with DWRG and DRG improved PSNR by 0.18 dB and FRM by 0.27 points on Set5.
• The combination of DWRG and DRG yielded the best performance (32.08 PSNR, 43.56 FRM on Set5), confirming the effectiveness of domain-specific architectures.
• Merging FreqNet’s output with GAN-based SR models like MSRResNet-GAN reduced artifacts while preserving high-frequency details, as shown in visual comparisons on the Manga109 dataset.
• The method enables practical post-processing enhancement of existing SR models by selectively replacing high-frequency DCT channels, improving fidelity without retraining.