[Paper Review] Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches

Xin Lin, Chao Ren+3 more|arXiv (Cornell University)|Aug 13, 2023

Image and Signal Denoising Methods4 citations

TL;DR

This paper proposes SCPGabNet, a self-collaborative parallel generative adversarial network for unsupervised real-world image denoising that enhances performance without increasing inference complexity. By iteratively replacing weaker denoisers with stronger ones in a noise extraction and denoising loop, the method generates better synthetic clean-noisy pairs, achieving state-of-the-art results on SIDD and DND benchmarks with a 1.86dB PSNR gain over baseline.

ABSTRACT

Deep learning methods have shown remarkable performance in image denoising, particularly when trained on large-scale paired datasets. However, acquiring such paired datasets for real-world scenarios poses a significant challenge. Although unsupervised approaches based on generative adversarial networks offer a promising solution for denoising without paired datasets, they are difficult in surpassing the performance limitations of conventional GAN-based unsupervised frameworks without significantly modifying existing structures or increasing the computational complexity of denoisers. To address this problem, we propose a SC strategy for multiple denoisers. This strategy can achieve significant performance improvement without increasing the inference complexity of the GAN-based denoising framework. Its basic idea is to iteratively replace the previous less powerful denoiser in the filter-guided noise extraction module with the current powerful denoiser. This process generates better synthetic clean-noisy image pairs, leading to a more powerful denoiser for the next iteration. This baseline ensures the stability and effectiveness of the training network. The experimental results demonstrate the superiority of our method over state-of-the-art unsupervised methods.

Motivation & Objective

• To address the challenge of unsupervised real-world image denoising without paired training data.
• To overcome performance limitations in existing GAN-based unsupervised denoising frameworks that cannot improve beyond initial training without structural changes or higher complexity.
• To develop a self-collaboration strategy that enables iterative model self-boosting using synthetic clean-noisy pairs.
• To design a stable and effective baseline framework with complementary 'self-synthesis' and 'unpaired-synthesis' constraints for robust training.

Proposed method

• Proposes a filter-guided noise extraction (NE) module that generates synthetic noisy images from reconstructed and noisy inputs.
• Introduces a parallel generative adversarial branches framework with two complementary branches: one for self-synthesis and one for unpaired-synthesis, enhancing training stability and diversity.
• Employs a self-collaboration (SC) strategy that iteratively replaces the previous denoiser with the current, more powerful one to refine synthetic data and boost performance.
• Uses a two-stage training process: first with patch size 112 and batch size 8, then fine-tuning with patch size 128 and batch size 4, both at a fixed 1e-4 learning rate.
• Leverages the NE module to extract noise from noisy images and the denoiser (DN) to reconstruct clean images, forming an R-B (replacement-boosting) iterative loop.
• Applies the SC strategy to multiple denoisers (e.g., DnCNN, UNet, DeamNet, DBSNL), demonstrating transferability across architectures.

Experimental results

Research questions

• RQ1 Can a self-collaboration mechanism improve GAN-based unsupervised image denoising without increasing inference complexity?
• RQ2 How effective is the proposed parallel generative adversarial branches framework with complementary self-synthesis and unpaired-synthesis constraints in stabilizing training and improving performance?
• RQ3 To what extent can the self-collaboration strategy enhance the performance of diverse denoising networks on real-world image denoising benchmarks?
• RQ4 Does the iterative refinement of synthetic clean-noisy pairs through SC lead to consistent and measurable PSNR gains across multiple iterations?
• RQ5 Can the SC strategy be generalized across different network architectures and datasets?

Key findings

• The proposed SCPGabNet achieves a 1.86dB PSNR improvement over the baseline PGabNet on the SIDD Benchmark, demonstrating the effectiveness of the self-collaboration strategy.
• The first iteration of the SC strategy yields a PSNR gain of over 0.5dB, with diminishing returns in later iterations, indicating rapid performance evolution.
• Applying the SC strategy to DnCNN improves PSNR by 0.96dB and SSIM by 0.0085 on the SIDD Validation set, confirming its transferability.
• UNet and DBSNL benefit from the SC strategy with 1.58dB and 1.52dB PSNR gains respectively, showing consistent performance gains across architectures.
• The method achieves state-of-the-art performance on both SIDD and DND benchmarks, outperforming existing unsupervised methods with large margins.
• The ablation study confirms that the combination of parallel branches with complementary constraints significantly improves training stability and final performance.