[Paper Review] Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy

Sreyas Mohan, Ramón Manzorro+7 more|arXiv (Cornell University)|Oct 24, 2020

Image and Signal Denoising Methods4 citations

TL;DR

This paper proposes a simulation-based denoising (SBD) framework using deep convolutional neural networks (CNNs) trained on synthetically generated transmission electron microscopy (TEM) images to address the lack of ground-truth clean data in scientific imaging. SBD significantly outperforms existing methods on both simulated and real low-signal-to-noise TEM data, demonstrating superior generalization and robustness, while introducing new metrics and likelihood-based visualization to assess scientific relevance beyond standard photographic benchmarks.

ABSTRACT

Denoising is a fundamental challenge in scientific imaging. Deep convolutional neural networks (CNNs) provide the current state of the art in denoising natural images, where they produce impressive results. However, their potential has barely been explored in the context of scientific imaging. Denoising CNNs are typically trained on real natural images artificially corrupted with simulated noise. In contrast, in scientific applications, noiseless ground-truth images are usually not available. To address this issue, we propose a simulation-based denoising (SBD) framework, in which CNNs are trained on simulated images. We test the framework on data obtained from transmission electron microscopy (TEM), an imaging technique with widespread applications in material science, biology, and medicine. SBD outperforms existing techniques by a wide margin on a simulated benchmark dataset, as well as on real data. Apart from the denoised images, SBD generates likelihood maps to visualize the agreement between the structure of the denoised image and the observed data. Our results reveal shortcomings of state-of-the-art denoising architectures, such as their small field-of-view: substantially increasing the field-of-view of the CNNs allows them to exploit non-local periodic patterns in the data, which is crucial at high noise levels. In addition, we analyze the generalization capability of SBD, demonstrating that the trained networks are robust to variations of imaging parameters and of the underlying signal structure. Finally, we release the first publicly available benchmark dataset of TEM images, containing 18,000 examples.

Motivation & Objective

• To address the challenge of denoising scientific images where ground-truth clean data is unavailable, particularly in electron microscopy.
• To develop a deep learning framework that generalizes well across varying imaging parameters and signal structures in scientific imaging.
• To identify limitations of existing photographic image denoising architectures when applied to scientific data such as TEM.
• To propose new evaluation metrics that reflect scientifically meaningful image fidelity rather than photographic quality.
• To release a public benchmark dataset of 18,000 TEM images for reproducible research in scientific image denoising.

Proposed method

• Training deep CNNs on a large simulated dataset of TEM images generated using physical models of electron beam interaction and noise processes.
• Using a simulation-based training paradigm where the network learns to reconstruct clean signals from noisy observations without requiring real ground-truth images.
• Employing a U-Net-like architecture with residual connections to enhance feature learning and gradient flow in low-SNR regimes.
• Introducing a likelihood map visualization technique based on forward modeling to assess the statistical consistency between denoised images and observed data.
• Designing domain-specific evaluation metrics—such as atomic column detection accuracy and structural preservation scores—tailored to atomic-resolution TEM.
• Applying gradient visualization and ablation studies to analyze feature learning and architectural sensitivity in scientific imaging contexts.

Experimental results

Research questions

• RQ1 Can deep CNNs trained on simulated scientific images generalize effectively to real-world low-signal-to-noise imaging data?
• RQ2 How do standard photographic image denoising architectures perform when applied to scientific imaging data such as TEM?
• RQ3 What metrics are more appropriate than PSNR and SSIM for evaluating scientific image denoising performance?
• RQ4 How does the choice of training data distribution affect the generalization and artifact profile of the denoised output?
• RQ5 Can likelihood-based visualization methods effectively flag regions of inconsistency between denoised images and raw observations?

Key findings

• The SBD framework outperforms state-of-the-art methods—including Fourier filtering, wavelet-based PURE-LET, and unsupervised denoising—on both simulated and real TEM datasets, especially at low signal-to-noise ratios.
• Standard photographic denoising architectures underperform on TEM data; expanding the receptive field (field-of-view) of the CNN significantly improves performance on low-SNR images.
• PSNR and SSIM fail to correlate with scientifically meaningful image quality; the proposed metrics such as atomic column detection accuracy and structural preservation score show stronger alignment with expert assessments.
• Training data mismatch leads to artifacts: networks trained on white-contrast simulations produce unnatural streak patterns in the bulk, while those trained on data matching real acquisition conditions recover structures more accurately.
• Unsupervised methods perform well on bulk regions but introduce significant artifacts on nanoparticle surfaces, highlighting a trade-off between background and feature fidelity.
• Likelihood maps effectively visualize regions of high and low consistency with observed data, helping to identify potential false positives such as phantom atoms, though they may fail in very low SNR regions.