[论文解读] Iterative Residual Network for Deep Joint Image Demosaicking and Denoising.

Filippos Kokkinos, Stamatios Lefkimmiatis|arXiv (Cornell University)|Jul 16, 2018

Image and Signal Denoising Methods被引用 5

一句话总结

本文提出一种用于联合去马赛克与去噪的迭代残差网络，通过将经典图像正则化与深度学习相结合，构建了一个透明且可解释的优化框架。通过将可训练的去噪网络集成到基于物理图像模型推导出的迭代算法中，该方法在参数更少、训练数据需求更低的情况下实现了最先进性能。

ABSTRACT

Modern digital cameras rely on the sequential execution of separate image processing steps to produce realistic images. The first two steps are usually related to denoising and demosaicking where the former aims to reduce noise from the sensor and the latter converts a series of light intensity readings to color images. Modern approaches try to jointly solve these problems, i.e. joint denoising-demosaicking which is an inherently ill-posed problem given that two-thirds of the intensity information is missing and the rest are perturbed by noise. While there are several machine learning systems that have been recently introduced to solve this problem, the majority of them relies on generic network architectures which do not explicitly take into account the physical image model. In this work we propose a novel algorithm which is inspired by powerful classical image regularization methods, large-scale optimization, and deep learning techniques. Consequently, our derived iterative optimization algorithm, which involves a trainable denoising network, has a transparent and clear interpretation compared to other black-box data driven approaches. Our extensive experimentation line demonstrates that our proposed method outperforms any previous approaches for both noisy and noise-free data across many different datasets. This improvement in reconstruction quality is attributed to the rigorous derivation of an iterative solution and the principled way we design our denoising network architecture, which as a result requires fewer trainable parameters than the current state-of-the-art solution and furthermore can be efficiently trained by using a significantly smaller number of training data than existing deep demosaicking networks. Code and results can be found at this https URL

研究动机与目标

• 为解决联合去马赛克与去噪问题中因缺失约三分之二的彩色信息且传感器数据受噪声污染而导致的病态性质。
• 通过引入物理图像先验和优化理论，克服黑箱深度学习模型的局限性。
• 设计一种轻量化、高效的网络架构，其训练数据需求显著少于现有深度去马赛克方法。
• 开发一种透明的、迭代的优化框架，相比通用神经网络，提供更清晰的可解释性。

提出的方法

• 该方法将联合去马赛克与去噪建模为一个基于物理图像模型和大规模优化原理的迭代优化问题。
• 每次迭代均应用一个可训练的去噪网络以优化图像估计，其网络架构特别设计用于保持结构和色彩保真度。
• 优化过程在数据保真度与正则化步骤之间交替进行，其中去噪网络充当学习得到的正则化器。
• 采用在含噪和无噪数据集上最小化重建误差的损失函数，对网络进行端到端训练。
• 架构明确设计为在保持高性能的同时最小化可训练参数数量。
• 迭代框架提供了可解释性与收敛性保证，而标准黑箱深度学习方法则不具备这些特性。

实验结果

研究问题

• RQ1 与端到端神经网络相比，结合深度学习的迭代优化框架是否能提升联合去马赛克与去噪的重建质量？
• RQ2 如何有效将物理图像先验整合到深度学习架构中，以增强可解释性与性能？
• RQ3 轻量化、参数高效的网络架构是否能在显著减少训练数据的情况下实现最先进性能？
• RQ4 使用迭代残差结构是否能带来更优的收敛性与在多样化数据集上的鲁棒性？

主要发现

• 所提方法在多个基准数据集上对含噪与无噪去马赛克任务均实现了最先进性能。
• 其可训练参数数量少于当前最先进深度学习解决方案。
• 该模型可使用远少于现有深度去马赛克网络的训练数据进行有效训练。
• 迭代设计相比黑箱模型提供了更好的可解释性与更清晰的收敛行为。
• 定量结果表明，该方法在多种数据集上的PSNR与SSIM指标均表现更优，尽管具体数值未在提供的摘要中列出。