[Paper Review] Unsupervised Image Denoising with Frequency Domain Knowledge

Nahyun Kim, Donggon Jang+3 more|arXiv (Cornell University)|Nov 29, 2021

Image and Signal Denoising Methods48 references4 citations

TL;DR

This paper proposes a GAN-based unsupervised image denoising method that leverages frequency domain knowledge via a spectral discriminator and frequency reconstruction loss to improve denoising performance without paired clean/noisy data. By focusing on high-frequency components where noise is most prominent, the method achieves state-of-the-art results on both synthetic and real-world datasets, outperforming existing unsupervised approaches and approaching supervised learning performance.

ABSTRACT

Supervised learning-based methods yield robust denoising results, yet they are inherently limited by the need for large-scale clean/noisy paired datasets. The use of unsupervised denoisers, on the other hand, necessitates a more detailed understanding of the underlying image statistics. In particular, it is well known that apparent differences between clean and noisy images are most prominent on high-frequency bands, justifying the use of low-pass filters as part of conventional image preprocessing steps. However, most learning-based denoising methods utilize only one-sided information from the spatial domain without considering frequency domain information. To address this limitation, in this study we propose a frequency-sensitive unsupervised denoising method. To this end, a generative adversarial network (GAN) is used as a base structure. Subsequently, we include spectral discriminator and frequency reconstruction loss to transfer frequency knowledge into the generator. Results using natural and synthetic datasets indicate that our unsupervised learning method augmented with frequency information achieves state-of-the-art denoising performance, suggesting that frequency domain information could be a viable factor in improving the overall performance of unsupervised learning-based methods.

Motivation & Objective

• Address the limitation of unsupervised image denoising methods that rely solely on spatial domain information, missing key statistical differences between clean and noisy images in the frequency domain.
• Overcome the dependency on large-scale paired clean/noisy datasets required by supervised methods, which are often unavailable or expensive to collect.
• Leverage the fact that noise is predominantly concentrated in high-frequency bands, while semantic content resides in low-frequency bands, to guide denoising.
• Introduce a novel framework that integrates frequency domain priors into a GAN-based architecture to improve image quality and perceptual plausibility.
• Demonstrate that frequency domain information can significantly enhance unsupervised denoising performance, making it competitive with supervised methods.

Proposed method

• Use a generative adversarial network (GAN) as the base architecture for unsupervised image denoising, with a generator mapping noisy images to denoised outputs.
• Introduce a spectral discriminator that specifically distinguishes between clean and denoised images in the high-frequency bands of the Fourier domain.
• Incorporate a frequency reconstruction loss that minimizes the difference between the Fourier transforms of the denoised image and the clean reference image.
• Combine the frequency reconstruction loss with cycle consistency loss to preserve image content and improve reconstruction fidelity.
• Train the generator to fool the spectral discriminator while minimizing frequency-domain reconstruction error, enabling it to focus on removing high-frequency noise.
• Use the texture discriminator to preserve fine details and contours, improving perceptual quality and SSIM, though with a trade-off in frequency-domain alignment.

Experimental results

Research questions

• RQ1 Can frequency domain information, particularly high-frequency band differences, be effectively leveraged to improve unsupervised image denoising?
• RQ2 Does integrating spectral discrimination and frequency reconstruction loss into a GAN framework lead to better denoising performance than spatial-only learning?
• RQ3 Can an unsupervised method using frequency priors achieve performance comparable to supervised methods trained on paired data?
• RQ4 How do the spectral discriminator and frequency reconstruction loss individually and jointly affect the balance between spatial and frequency domain fidelity?
• RQ5 Is the proposed method robust to real-world noise patterns, as well as synthetic additive white Gaussian noise?

Key findings

• The proposed method achieves a PSNR of 34.30 dB and SSIM of 0.9334 on the SIDD dataset, outperforming all unsupervised baselines and approaching supervised performance.
• The spectral discriminator alone improves PSNR by 0.2 dB and reduces LFD (frequency-domain distance) from 6.5955 to 6.5649, confirming its effectiveness in noise removal on high-frequency bands.
• Adding the frequency reconstruction loss reduces LFD to 6.5795 and increases PSNR to 26.03 dB and SSIM to 0.8375 in the ablation study, showing its role in balancing spatial and frequency domain reconstruction.
• The texture discriminator improves SSIM by 0.0041 (to 0.8375) but slightly increases LFD, indicating a trade-off between spatial detail preservation and frequency-domain accuracy.
• The full model with all components (spectral discriminator, texture discriminator, and frequency reconstruction loss) achieves the best overall performance, with the lowest LFD and highest PSNR/SSIM.
• Qualitative results show that the proposed method effectively removes intense noise while preserving color and fine textures, outperforming CBM3D, RedNet-30, DIP, and LIR in visual quality.