[Paper Review] Real-world Noisy Image Denoising: A New Benchmark

Jun Xu, Hui Li+3 more|arXiv (Cornell University)|Apr 7, 2018

Image and Signal Denoising Methods24 references174 citations

TL;DR

This paper introduces a large real-world noisy image denoising benchmark captured across multiple cameras and settings, evaluates existing methods, and shows real-world noise differs from AWGN with new methods offering robustness.

ABSTRACT

Most of previous image denoising methods focus on additive white Gaussian noise (AWGN). However,the real-world noisy image denoising problem with the advancing of the computer vision techiniques. In order to promote the study on this problem while implementing the concurrent real-world image denoising datasets, we construct a new benchmark dataset which contains comprehensive real-world noisy images of different natural scenes. These images are captured by different cameras under different camera settings. We evaluate the different denoising methods on our new dataset as well as previous datasets. Extensive experimental results demonstrate that the recently proposed methods designed specifically for realistic noise removal based on sparse or low rank theories achieve better denoising performance and are more robust than other competing methods, and the newly proposed dataset is more challenging. The constructed dataset of real photographs is publicly available at \url{https://github.com/csjunxu/PolyUDataset} for researchers to investigate new real-world image denoising methods. We will add more analysis on the noise statistics in the real photographs of our new dataset in the next version of this article.

Motivation & Objective

• Motivate real-world noise removal beyond synthetic AWGN denoising benchmarks.
• Create a large, diverse real-world noisy image dataset with corresponding ground-truth images.
• Assess existing denoising methods on real-world data and analyze robustness across cameras and ISO settings.

Proposed method

• Capture 500 images per scene across multiple cameras and settings to model real-world noise.
• Generate ground-truth by averaging non-misaligned, luminance-consistent frames per scene.
• Crop 512x512 patches from large images for practical denoising evaluation.
• Evaluate a wide range of denoising methods including AWGN-focused, grey-channel, discriminative, and real-world tailored approaches.
• Compare performance using PSNR and SSIM across datasets and analyze noise statistics by ISO and color channels.

Experimental results

Research questions

• RQ1 How well do existing denoising methods, including those designed for AWGN, perform on real-world noisy images across diverse cameras and settings?
• RQ2 Do methods tailored for real-world noise (e.g., Guided, MCWNNM, TWSC) outperform traditional AWGN-denoising methods on this benchmark?
• RQ3 How does denoising performance vary with ISO, camera brand, and scene content in real-world data?
• RQ4 What are the statistics of real-world noise across RGB channels as ISO changes?

Key findings

• Real-world noise is not well modeled by AWGN; methods designed for real-world noise outperform AWGN-focused baselines.
• Cross-channel methods (processing RGB channels together) generally yield better results than channel-wise gray-scale approaches.
• Discriminative learning methods trained on paired data show limited generalization when noise statistics differ from training data.
• Recent real-world tailored methods (Guided, MCWNNM, TWSC) achieve strong performance on some datasets but show smaller margins on the newly proposed benchmark, indicating higher dataset difficulty.
• The proposed dataset is more comprehensive and challenging, highlighting the need for new real-world denoising approaches.