[Paper Review] NBNet: Noise Basis Learning for Image Denoising with Subspace Projection

Shen Cheng, Yuzhi Wang+4 more|arXiv (Cornell University)|Dec 30, 2020

Image and Signal Denoising Methods52 references17 citations

TL;DR

NBNet proposes a novel image denoising framework that leverages subspace projection to enhance signal recovery by learning noise-robust basis vectors in feature space. By introducing a non-local attention module (SSA) for end-to-end basis generation and projection, NBNet achieves state-of-the-art PSNR and SSIM on SIDD and DND benchmarks with significantly lower computational cost than prior methods.

ABSTRACT

In this paper, we introduce NBNet, a novel framework for image denoising. Unlike previous works, we propose to tackle this challenging problem from a new perspective: noise reduction by image-adaptive projection. Specifically, we propose to train a network that can separate signal and noise by learning a set of reconstruction basis in the feature space. Subsequently, image denosing can be achieved by selecting corresponding basis of the signal subspace and projecting the input into such space. Our key insight is that projection can naturally maintain the local structure of input signal, especially for areas with low light or weak textures. Towards this end, we propose SSA, a non-local subspace attention module designed explicitly to learn the basis generation as well as the subspace projection. We further incorporate SSA with NBNet, a UNet structured network designed for end-to-end image denosing. We conduct evaluations on benchmarks, including SIDD and DND, and NBNet achieves state-of-the-art performance on PSNR and SSIM with significantly less computational cost.

Motivation & Objective

• Address the challenge of preserving fine textures and low-light details in image denoising, where conventional CNNs struggle due to reliance on local filters.
• Overcome the limitations of local response-based denoising in low signal-to-noise ratio (SNR) regions by incorporating global structural information.
• Develop an efficient, end-to-end trainable mechanism that learns signal subspace basis and performs projection without adding complex convolutional blocks.
• Demonstrate that subspace projection is a promising alternative to complex network architectures or explicit noise modeling in image denoising.
• Achieve superior performance with reduced computational overhead compared to existing SOTA methods on standard benchmarks.

Proposed method

• Propose a novel denoising framework based on image-adaptive subspace projection, where clean signal is reconstructed by projecting noisy features onto a learned signal subspace.
• Design the Subspace Attention (SSA) module to jointly learn basis vectors and perform projection using non-local attention, enabling global context modeling.
• Integrate SSA into a UNet-like architecture to preserve skip-connections and spatial details, with basis generation informed by features from both encoder and skip-connection paths.
• Formulate the projection operation as $\text{Proj}(\mathbf{X}_1, \mathbf{X}_2) = \mathbf{X}_1^T \mathbf{B} (\mathbf{B}^T \mathbf{B})^{-1} \mathbf{B}^T \mathbf{X}_1$, where $\mathbf{B}$ is the learned basis from $\mathbf{X}_2$.
• Train the entire network end-to-end using a standard L2 loss between predicted and clean images, with SSA enabling efficient, differentiable projection.
• Use a compact basis dimension $K=8$ or $16$, which is shown to be robust and sufficient for capturing essential signal structures without overfitting.

Experimental results

Research questions

• RQ1 Can subspace projection be effectively used to enhance image denoising by preserving local structures in low-SNR regions?
• RQ2 How does learning basis vectors via non-local attention improve denoising performance compared to standard convolutional operations?
• RQ3 What is the optimal configuration for basis generation and projection in terms of feature source and dimensionality?
• RQ4 Can subspace-based denoising achieve SOTA performance with lower computational cost than deep CNNs with additional blocks?
• RQ5 How does the proposed method compare to traditional and deep learning-based baselines in preserving weak textures and high-frequency details?

Key findings

• NBNet achieves a PSNR of 39.75 dB and SSIM of 0.973 on the SIDD benchmark, outperforming all prior methods including VDN, DANet, MIRNet, and CBDNet.
• On the DND benchmark, NBNet achieves state-of-the-art performance with superior texture and sharpness preservation, especially in weak-textured and low-light regions.
• The SSA module improves PSNR by 1.15 dB over DnCNN and 0.46 dB over DnCNN with SSA, demonstrating its effectiveness even when added to a strong baseline.
• The ablation study shows that projecting $\mathbf{X}_1$ onto a basis generated from both $\mathbf{X}_1$ and $\mathbf{X}_2$ yields the best performance (39.75 dB PSNR), outperforming single-source basis generation.
• Basis visualization confirms that the learned 16-channel basis captures repeating patterns (e.g., dots), enabling consistent texture recovery across the image via global correlation.
• The method maintains high performance with only a small computational overhead, achieving SOTA results while being more efficient than adding extra convolutional layers.