[Paper Review] Boosting CNN beyond Label in Inverse Problems

Eunju Cha, Jaeduck Jang+3 more|arXiv (Cornell University)|Jun 18, 2019

Image and Signal Denoising Methods32 references4 citations

TL;DR

This paper proposes Noise2Boosting, a novel boosting estimator for unbiased prediction error estimation in convolutional neural networks (CNNs) applied to inverse problems. By leveraging combinatorial convolutional frame representation and batch normalization, it enables end-to-end training beyond labels—achieving superior performance in denoising, super-resolution, and MRI reconstruction without requiring clean labels or multiple noisy realizations.

ABSTRACT

Convolutional neural networks (CNN) have been extensively used for inverse problems. However, their prediction error for unseen test data is difficult to estimate a priori since the neural networks are trained using only selected data and their architecture are largely considered a blackbox. This poses a fundamental challenge to neural networks for unsupervised learning or improvement beyond the label. In this paper, we show that the recent unsupervised learning methods such as Noise2Noise, Stein's unbiased risk estimator (SURE)-based denoiser, and Noise2Void are closely related to each other in their formulation of an unbiased estimator of the prediction error, but each of them are associated with its own limitations. Based on these observations, we provide a novel boosting estimator for the prediction error. In particular, by employing combinatorial convolutional frame representation of encoder-decoder CNN and synergistically combining it with the batch normalization, we provide a close form formulation for the unbiased estimator of the prediction error that can be minimized for neural network training beyond the label. Experimental results show that the resulting algorithm, what we call Noise2Boosting, provides consistent improvement in various inverse problems under both supervised and unsupervised learning setting.

Motivation & Objective

• To address the challenge of estimating prediction error in CNNs for inverse problems when no clean labels are available.
• To overcome limitations of existing unsupervised methods like Noise2Noise (requires multiple noisy realizations) and Noise2Void (limited by subsampling).
• To develop a generalizable, label-free training framework that improves generalization beyond the label using an explicit, closed-form unbiased estimator of prediction error.
• To enable robust training of encoder-decoder CNNs (e.g., U-Net) in both supervised and unsupervised settings using only a single noisy input per image.

Proposed method

• Proposes a combinatorial convolutional frame representation of encoder-decoder CNNs to model input-dependent linear frame structures.
• Derives a closed-form unbiased estimator of prediction error using the divergence of the network’s output with respect to input, based on ReLU network properties.
• Introduces a boosting strategy that combines multiple outputs from randomly weighted or masked input samples to reduce prediction error.
• Employs batch normalization to trivialize the divergence term’s contribution, simplifying training and avoiding identity mapping learning.
• Uses random weighting (0.8–1.2) and noise augmentation (σ ∈ [10,40]) on noisy inputs to generate diverse training samples without requiring clean images.
• Applies the derived estimator as a loss function during training, enabling end-to-end optimization without ground-truth labels.

Experimental results

Research questions

• RQ1 Can an unbiased estimator of prediction error be derived in closed form for ReLU-based encoder-decoder CNNs?
• RQ2 How can the prediction error estimator be leveraged to train CNNs beyond labels in inverse problems?
• RQ3 Can the limitations of Noise2Noise (need for multiple noisy realizations) and Noise2Void (subsampling-induced performance loss) be overcome?
• RQ4 Does combining multiple boosted network outputs via random input weighting improve generalization and image quality?
• RQ5 Can the proposed method achieve state-of-the-art performance in unsupervised inverse problems with only a single noisy input per image?

Key findings

• Noise2Boosting outperforms Noise2Noise in PSNR and SSIM on Gaussian denoising (σ=25), achieving 29.117 PSNR and 0.742 SSIM on Set5, compared to 29.180 and 0.756 for Noise2Noise.
• The method achieves better texture and detail recovery than Noise2Noise, as visually confirmed in Figure 5 for denoising with σ=25.
• Noise2Boosting achieves competitive performance in super-resolution, accelerated MRI, and electron microscopy, demonstrating generalizability across inverse problems.
• The use of random input weighting and noise augmentation enables effective data augmentation without requiring clean images or multiple noisy realizations.
• Theoretical analysis confirms that the divergence term’s contribution is minimized via batch normalization, enabling stable and efficient training.
• The proposed framework enables training CNNs in unsupervised settings with a single noisy input, significantly reducing data acquisition constraints.