[Paper Review] DivNoising: Diversity Denoising with Fully Convolutional Variational Autoencoders.

Mangal Prakash, Alexander Krull+1 more|arXiv (Cornell University)|Jun 10, 2020

Cell Image Analysis Techniques9 citations

TL;DR

DivNoising introduces a fully convolutional variational autoencoder that predicts a diverse distribution of denoised images rather than a single output, enabling uncertainty-aware restoration in microscopy. By sampling from the posterior distribution, it improves downstream tasks like OCR and cell segmentation without requiring paired clean data.

ABSTRACT

Deep Learning based methods have emerged as the indisputable leaders for virtually all image restoration tasks. Especially in the domain of microscopy images, various content-aware image restoration (CARE) approaches are now used to improve the interpretability of acquired data. But there are limitations to what can be restored in corrupted images, and any given method needs to make a sensible compromise between many possible clean signals when predicting a restored image. Here, we propose DivNoising -- a denoising approach based on fully-convolutional variational autoencoders, overcoming this problem by predicting a whole distribution of denoised images. Our method is unsupervised, requiring only noisy images and a description of the imaging noise, which can be measured or bootstrapped from noisy data. If desired, consensus predictions can be inferred from a set of DivNoising predictions, leading to competitive results with other unsupervised methods and, on occasion, even with the supervised state-of-the-art. DivNoising samples from the posterior enable a plethora of useful applications. We are (i) discussing how optical character recognition (OCR) applications could benefit from diverse predictions on ambiguous data, and (ii) show in detail how instance cell segmentation gains performance when using diverse DivNoising predictions.

Motivation & Objective

• To address the limitation of single-output denoising methods that fail to capture uncertainty in corrupted microscopy images.
• To develop an unsupervised denoising framework that leverages only noisy images and noise model descriptions.
• To enable diverse predictions that improve performance in downstream tasks such as optical character recognition and instance cell segmentation.
• To provide a distributional output that reflects uncertainty, supporting robust decision-making in image analysis.

Proposed method

• The method employs a fully convolutional variational autoencoder (VAE) to model the posterior distribution over clean images given a noisy input.
• It uses reparameterization trick for end-to-end training, enabling backpropagation through stochastic sampling from the posterior.
• Noise characteristics are modeled via a known or estimated noise model, which is used to condition the VAE's inference and generative networks.
• Diverse denoised outputs are generated by sampling multiple times from the posterior distribution of the trained VAE.
• Consensus predictions are computed from multiple samples to improve robustness and performance.
• The approach is entirely unsupervised, requiring no paired clean-noisy image pairs for training.

Experimental results

Research questions

• RQ1 Can a variational autoencoder framework generate diverse, high-quality denoised outputs from noisy microscopy images without paired training data?
• RQ2 How does sampling from a distribution of denoised images improve performance in downstream tasks like OCR and cell segmentation?
• RQ3 To what extent can consensus predictions from diverse samples match or exceed supervised state-of-the-art results?
• RQ4 How does the method handle uncertainty in image restoration when the true clean signal is ambiguous?

Key findings

• DivNoising achieves competitive performance compared to other unsupervised denoising methods, with results approaching those of supervised state-of-the-art models.
• The use of diverse predictions significantly improves instance cell segmentation performance, demonstrating the value of uncertainty-aware outputs.
• OCR applications benefit from diverse predictions by handling ambiguous or low-contrast text in noisy images more robustly.
• Consensus predictions derived from multiple samples yield stable and high-quality restorations, reducing variance in output.
• The method operates effectively with only noisy images and a noise model description, eliminating the need for paired data.