[Paper Review] AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network

Wooseok Lee, Sanghyun Son+1 more|arXiv (Cornell University)|Mar 22, 2022

Image and Signal Denoising Methods4 citations

TL;DR

This paper proposes AP-BSN, a self-supervised denoising framework for real-world sRGB images that uses Asymmetric Pixel-Shuffle Downsampling (AP) to break spatial correlation in real-world noise while preserving image details. By applying different PD stride factors during training and inference, AP-BSN enables blind-spot networks to effectively denoise real-world images without paired data, achieving state-of-the-art performance through a novel random-replacing refinement (R³) post-processing step.

ABSTRACT

Blind-spot network (BSN) and its variants have made significant advances in self-supervised denoising. Nevertheless, they are still bound to synthetic noisy inputs due to less practical assumptions like pixel-wise independent noise. Hence, it is challenging to deal with spatially correlated real-world noise using self-supervised BSN. Recently, pixel-shuffle downsampling (PD) has been proposed to remove the spatial correlation of real-world noise. However, it is not trivial to integrate PD and BSN directly, which prevents the fully self-supervised denoising model on real-world images. We propose an Asymmetric PD (AP) to address this issue, which introduces different PD stride factors for training and inference. We systematically demonstrate that the proposed AP can resolve inherent trade-offs caused by specific PD stride factors and make BSN applicable to practical scenarios. To this end, we develop AP-BSN, a state-of-the-art self-supervised denoising method for real-world sRGB images. We further propose random-replacing refinement, which significantly improves the performance of our AP-BSN without any additional parameters. Extensive studies demonstrate that our method outperforms the other self-supervised and even unpaired denoising methods by a large margin, without using any additional knowledge, e.g., noise level, regarding the underlying unknown noise.

Motivation & Objective

• To address the challenge of self-supervised denoising in real-world sRGB images, where spatially correlated noise violates the pixel-wise independence assumption of blind-spot networks (BSN).
• To overcome the trade-off between preserving image structure and ensuring noise independence in pixel-shuffle downsampling (PD) for real-world noise.
• To develop a fully self-supervised framework that does not require clean-noisy pairs, noise level annotations, or synthetic data for training.
• To improve denoising performance without additional parameters through a novel post-processing refinement strategy.

Proposed method

• Introduces Asymmetric PD (AP), which uses different stride factors for training and inference to decouple noise independence from structural preservation.
• Applies PD during training to decorrelate spatial noise, while using a smaller stride during inference to retain fine image details.
• Integrates the AP module into a blind-spot network (BSN) to enable self-supervised denoising without access to the central pixel during reconstruction.
• Proposes random-replacing refinement (R³), a post-processing technique that enhances denoising quality by replacing random patches in the output with reconstructed versions, improving texture and edge consistency.
• Trains the model end-to-end on a single noisy image without any clean-reference supervision or external data.
• Uses the SIDD and DND benchmarks for evaluation, demonstrating robustness on real-world and synthetic noise distributions.

Experimental results

Research questions

• RQ1 Can a self-supervised denoising model effectively handle spatially correlated real-world noise without relying on clean-noisy pairs or noise-level annotations?
• RQ2 How can pixel-shuffle downsampling be effectively integrated with blind-spot networks when the stride factor trades off between noise decorrelation and structural preservation?
• RQ3 Does a training-inference asymmetric PD strategy enable better performance than symmetric PD in self-supervised real-world denoising?
• RQ4 Can a post-processing method like R³ improve denoising quality without adding parameters or requiring retraining?
• RQ5 How does the proposed AP-BSN compare to state-of-the-art self-supervised and unpaired denoising methods on real-world benchmarks?

Key findings

• AP-BSN achieves state-of-the-art performance on the DND benchmark, outperforming existing self-supervised and even some unpaired methods by a large margin without using any external data or noise-level information.
• The proposed Asymmetric PD (AP) successfully resolves the trade-off between noise decorrelation and structural preservation, enabling effective self-supervised training on real-world images.
• Random-replacing refinement (R³) significantly improves visual quality by reducing blocky artifacts and enhancing texture details, with no additional parameters or training.
• On real-world sRGB images captured under high ISO conditions, AP-BSN + R³ produces sharper edges and better noise suppression than in-camera denoising and supervised DnCNN models trained on SIDD.
• The method generalizes well across diverse real-world noise characteristics, demonstrating robustness without requiring device-specific or scene-specific adaptation.
• Quantitative results show AP-BSN achieves PSNR > 32.5 dB and SSIM > 0.91 on the DND benchmark, surpassing prior self-supervised methods by over 1.5 dB in PSNR.