[论文解读] Scaling Laws For Deep Learning Based Image Reconstruction

Tobit Klug, Reinhard Heckel|arXiv (Cornell University)|Sep 27, 2022

Photoacoustic and Ultrasonic Imaging被引用 22

一句话总结

本论文研究深度学习模型的尺度属性如何影响图像重建性能，旨在建立实用的尺度趋势与指南。

ABSTRACT

Deep neural networks trained end-to-end to map a measurement of a (noisy) image to a clean image perform excellent for a variety of linear inverse problems. Current methods are only trained on a few hundreds or thousands of images as opposed to the millions of examples deep networks are trained on in other domains. In this work, we study whether major performance gains are expected from scaling up the training set size. We consider image denoising, accelerated magnetic resonance imaging, and super-resolution and empirically determine the reconstruction quality as a function of training set size, while simultaneously scaling the network size. For all three tasks we find that an initially steep power-law scaling slows significantly already at moderate training set sizes. Interpolating those scaling laws suggests that even training on millions of images would not significantly improve performance. To understand the expected behavior, we analytically characterize the performance of a linear estimator learned with early stopped gradient descent. The result formalizes the intuition that once the error induced by learning the signal model is small relative to the error floor, more training examples do not improve performance.

研究动机与目标

• 通过研究尺度对深度学习模型中图像重建质量的影响来激发本研究。
• 刻画经验尺度趋势，以指导重建任务的模型设计与数据需求。
• 提供实用洞见，指引在何时增加模型容量或数据以提升重建效果。

提出的方法

• 调查或分析应用于图像重建任务的深度学习模型的尺度行为。
• 识别并讨论影响尺度的关键因素，如模型规模、数据可用性和优化动态。
• 将研究结果综合成可操作的指南，供设计重建系统的从业者使用。

实验结果

研究问题

• RQ1 在深度学习框架中，模型容量的变化如何影响图像重建性能？
• RQ2 训练数据的数量和质量对重建任务的尺度行为有何影响？
• RQ3 可以为在图像重建问题中选择模型规模和数据配置得出哪些实用指南？
• RQ4 随着模型规模扩大用于重建目标时，性能是否存在边际收益递减或阶段性转变？

主要发现

• 出现了将模型规模与重建质量以可预测方式相关联的尺度趋势。
• 数据可用性与优化选择影响增大容量在多大程度上转化为性能提升。
• 可以形成在何时扩大模型容量与收集更多数据以用于重建任务的指南。
• 在某些重建设定下，扩展规模带来的效益可能存在实际限制或阶段性转变。