[论文解读] Noise2Self: Blind Denoising by Self-Supervision

Joshua Batson, Löıc A. Royer|arXiv (Cornell University)|Jan 30, 2019

Cell Image Analysis Techniques被引用 120

一句话总结

该论文提出 Noise2Self，一种自监督、J-invariant 去噪方法，通过掩蔽输入的部分并预测被掩蔽的值来实现去噪，得到高斯过程理论支撑并有基于 UNet 的实际实验。

ABSTRACT

We propose a general framework for denoising high-dimensional measurements which requires no prior on the signal, no estimate of the noise, and no clean training data. The only assumption is that the noise exhibits statistical independence across different dimensions of the measurement, while the true signal exhibits some correlation. For a broad class of functions ("$\mathcal{J}$-invariant"), it is then possible to estimate the performance of a denoiser from noisy data alone. This allows us to calibrate $\mathcal{J}$-invariant versions of any parameterised denoising algorithm, from the single hyperparameter of a median filter to the millions of weights of a deep neural network. We demonstrate this on natural image and microscopy data, where we exploit noise independence between pixels, and on single-cell gene expression data, where we exploit independence between detections of individual molecules. This framework generalizes recent work on training neural nets from noisy images and on cross-validation for matrix factorization.

研究动机与目标

• Motivate and develop a denoising method that does not require clean ground truth data.
• Provide theoretical guarantees comparing self-supervised predictors to Gaussian counterparts.
• Propose practical masking schemes and trainable architectures to realize J-invariant denoisers.
• Demonstrate effectiveness on diverse datasets using neural networks and calibration analyses.

提出的方法

• Formulate J-invariant self-supervision where the predictor uses masked/interpolated inputs to predict masked coordinates.
• Introduce g(x)_J := f_theta(1_J^c * x_J + 1_J * s(x))_J, with s(x) as a local interpolation (e.g., a fixed kernel).
• Present Gaussian-process based lemmas and propositions showing that, under Gaussian assumptions, the optimal J-invariant predictor is no worse than the Gaussian case and that conditioning on x_J^c yields a bound on the MSE.
• Discuss masking strategies (Uniform Pixel Selection) and the issue of information leakage, advocating truly J-invariant schemes where the masked value does not influence the prediction.
• Propose and analyze linear combinations of uncorrelated unbiased estimators to potentially improve PSNR when combining the self-supervised predictor with the original noisy input.
• Describe training with a randomized J (subset of coordinates) and a UNet architecture, plus inference options with true J-invariance or direct full-input application

实验结果

研究问题

• RQ1 Can denoising be effectively learned without clean targets using a J-invariant self-supervised setup?
• RQ2 How does the self-supervised predictor compare to Gaussian baselines in terms of mean-squared error?
• RQ3 What masking/interpolation strategies best realize true J-invariance and maximize denoising performance?
• RQ4 Is it beneficial to linearly combine the self-supervised predictor with the original noisy input, and how does this affect PSNR?

主要发现

• A Gaussian-process based result shows that the self-supervised J-invariant predictor is competitive with or better than the corresponding Gaussian predictor. (Proposition 1)
• Under certain conditions, combining the self-supervised estimator with the raw input using an optimal linear mix can reduce variance and potentially improve PSNR. (Section 3.2)
• Uniform Pixel Selection masking can bias the predictor unless care is taken to ensure J-invariance; the authors propose excluding the value being predicted when estimating local distributions.
• Training with randomized J and a UNet can produce genuinely J-invariant predictions, and inference with full input can yield about 0.5 dB PSNR improvement over the strictly J-invariant version on the evaluated datasets.
• The authors calibrate traditional denoisers (median, wavelet, NL-means) using self-supervised loss to select optimal parameters without ground-truth references (calibration plots).
• Experiments on Hànzì, CellNet, and ImageNet with a UNet demonstrate practical self-supervised denoising across synthetic and real-noise models.