[论文解读] Duke PAM Dataset

Anthony DiSpirito, Daiwei Li+6 more|arXiv (Cornell University)|May 30, 2020

Photoacoustic and Ultrasonic Imaging参考文献 19被引用 15

一句话总结

本文提出一种基于深度学习的方法，采用全连接U-Net（FD U-Net）重建欠采样的光声显微镜（PAM）图像，克服了成像速度与空间分辨率之间的权衡。该模型仅需原始像素的2%即可实现高质量重建，显著缩短成像时间，同时保持图像质量。

ABSTRACT

Duke PAM is a photoacoustic microscopy (PAM) dataset collected at Duke University with the optical resolution PAM system described in (M. Chen et al., "Simultaneous photoacoustic imaging of intravascular and tissue oxygenation," <em>Optics Letters</em>, vol. 44, no. 15, pp. 3773-3776, 2019.) at a wavelength of 532 nm. This dataset is composed of primarily mouse brain microvasculature images, with a few images of mouse ear and tumors. Although collected in 3-D, for the purposes of viewing, collection, and storage, the images have been projected into 2-D using maximum amplitude projection (MAP). These data were collected with support from the National Institutes of Health (R01 EB028143, R01 NS111039, R01 NS115581, R21 EB027304, R43 CA243822, R43 CA239830, R44 HL138185); Duke MEDx Basic Science Grant; Duke Center for Genomic and Computational Biology Faculty Research Grant; Duke Institute of Brain Science Incubator Award; American Heart Association Collaborative Sciences Award (18CSA34080277). The "clean" subset contains PAM images of variable size that have been preprocessed and stored as uint8 .jpg images. The "patches" subset contains non-overlapping 128-by-128 pixel patches of the "clean" images stored as uint8 .jpg images. The "raw" subset contains PAM images of variable size before they have been preprocessed (stored as uint16 .png images). For general deep learning tasks, the "clean" dataset can be used with random crop (none of the images have dimensions less than 128-by-128). If uniformly-sized images are desired without needing to use random crop, the "patches" subset would work best. The "raw" subset works best for deep learning tasks that expect input images that are closer to raw/unprocessed PAM images and are willing to take on the memory burden of loading uint16 .png images.

研究动机与目标

• 为克服光声显微镜（PAM）中空间分辨率与成像速度之间的权衡。
• 开发一种深度学习模型，能够从严重欠采样的数据中重建出高质量的PAM图像。
• 评估模型在不同欠采样水平下的鲁棒性，以模拟实际成像约束条件。
• 利用人工下采样的全采样PAM图像（小鼠脑血管）建立重建性能的基准真实值。

提出的方法

• 在多个比例下对小鼠脑血管的全采样PAM图像进行人工下采样，以模拟欠采样条件。
• 训练并评估多种卷积神经网络（CNN）架构，最终选择全连接U-Net（FD U-Net）以实现最佳性能。
• 采用FD U-Net架构学习从低分辨率、欠采样PAM输入到高分辨率、重建输出的映射关系。
• 在合成的欠采样数据上端到端训练模型，使用原始全采样图像作为真实值。
• 在未见过的欠采样数据上进行推理，以评估不同采样比例下的重建保真度。
• 通过数值指标和视觉分析评估性能，以量化图像质量的保持程度。

实验结果

研究问题

• RQ1 深度学习能否有效从原始像素数据的2%以内重建光声显微镜图像？
• RQ2 FD U-Net在重建欠采样PAM图像方面的性能与其它CNN架构相比如何？
• RQ3 在极端欠采样条件下，该方法在多大程度上能保持图像质量和结构保真度？
• RQ4 该模型在不同成像条件下对不同水平的欠采样具有多强的鲁棒性？

主要发现

• FD U-Net模型仅使用原始像素的2%即可实现高质量的PAM图像重建，展现出卓越的数据效率。
• 即使在极端欠采样比例下，该模型仍能保持重建图像的结构和纹理保真度。
• 数值分析证实，模型在多个采样水平下均表现稳定，图像质量退化极小。
• 在视觉和定量指标上，FD U-Net均优于其他测试的CNN架构，包括标准U-Net和残差网络。
• 该方法可显著缩短成像时间——采集速度可能提升至原来的50倍，同时分辨率或诊断质量无明显损失。