[Paper Review] Deep Class Aware Denoising

Tal Remez, Or Litany+2 more|arXiv (Cornell University)|Jan 6, 2017

Image and Signal Denoising Methods52 references7 citations

TL;DR

This paper proposes a class-aware deep denoising framework that improves image denoising performance by fine-tuning a convolutional neural network (DenoiseNet) on specific semantic image classes, such as faces or flowers. By leveraging pre-trained image classifiers and adapting the denoiser per class, the method achieves up to 0.4 dB PSNR gain over class-agnostic baselines, demonstrating that semantic awareness significantly enhances reconstruction quality.

ABSTRACT

The increasing demand for high image quality in mobile devices brings forth the need for better computational enhancement techniques, and image denoising in particular. At the same time, the images captured by these devices can be categorized into a small set of semantic classes. However simple, this observation has not been exploited in image denoising until now. In this paper, we demonstrate how the reconstruction quality improves when a denoiser is aware of the type of content in the image. To this end, we first propose a new fully convolutional deep neural network architecture which is simple yet powerful as it achieves state-of-the-art performance even without being class-aware. We further show that a significant boost in performance of up to $0.4$ dB PSNR can be achieved by making our network class-aware, namely, by fine-tuning it for images belonging to a specific semantic class. Relying on the hugely successful existing image classifiers, this research advocates for using a class-aware approach in all image enhancement tasks.

Motivation & Objective

• To address the gap in image denoising where semantic content class is not exploited despite its prevalence in mobile photography.
• To investigate whether improving denoising performance is possible by tailoring the denoiser to specific semantic image classes.
• To demonstrate that fine-tuning a general-purpose denoiser on class-specific data yields significant performance gains.
• To explore the feasibility of using off-the-shelf image classifiers to enable class-aware image enhancement.
• To validate that state-of-the-art image classifiers remain robust even under high noise, enabling reliable class assignment for denoising.

Proposed method

• A fully convolutional deep neural network (DenoiseNet) is proposed as a strong baseline denoiser, achieving state-of-the-art performance without class awareness.
• The network is fine-tuned on images from specific semantic classes (e.g., faces, flowers) using class-specific training data.
• Pre-trained image classifiers (e.g., ResNet) are used to assign semantic labels to input images, enabling class-aware adaptation.
• The denoising process is gradual: each layer estimates and removes a portion of the noise, with shallow layers handling flat regions and deeper layers focusing on edges and textures.
• Noise estimation is visualized across layers, showing that deeper layers contribute more to edge and contour recovery.
• A layer-wise contribution map is generated to identify which layer most significantly alters each pixel’s value during denoising.

Experimental results

Research questions

• RQ1 Can performance in image denoising be improved by incorporating semantic class information into the denoising process?
• RQ2 To what extent does fine-tuning a general-purpose denoiser on class-specific data improve PSNR and visual quality?
• RQ3 Does the gradual noise removal behavior observed in the network align with known principles of effective denoising?
• RQ4 How do different network layers contribute to noise estimation and image reconstruction across semantic classes?
• RQ5 Can off-the-shelf image classifiers be reliably used to assign semantic classes for the purpose of class-aware denoising?

Key findings

• Fine-tuning the denoiser on class-specific data leads to a significant performance boost of up to 0.4 dB PSNR over the class-agnostic baseline.
• The class-aware denoiser produces visually superior results, reducing artifacts commonly introduced by general-purpose denoisers, especially in complex textures and facial features.
• The denoising process is inherently gradual: noise is removed progressively across layers, with shallow layers handling flat regions and deeper layers focusing on edges and contours.
• Despite no explicit training objective for monotonic error reduction, the reconstruction error consistently decreases with increasing layer depth across all test images.
• The first few layers dominate the denoising process for most pixels, while deeper layers refine edges and textures, indicating a hierarchical processing strategy.
• The network’s internal noise estimation across layers reveals distinct roles: early layers detect local noise, while deeper layers capture global structural patterns.