[Paper Review] Photoacoustic Microscopy with Sparse Data Enabled by Convolutional Neural Networks for Fast Imaging

Jiasheng Zhou, Da He+4 more|arXiv (Cornell University)|Jun 8, 2020

Photoacoustic and Ultrasonic Imaging32 references4 citations

TL;DR

This paper proposes a convolutional neural network (CNN)-based method to reconstruct high-quality photoacoustic microscopy (PAM) images from sparse data, enabling faster imaging by reducing scanning points. By leveraging residual blocks, squeeze-and-excitation modules, and a perceptual loss function, the model restores 1/4 or 1/16 sampled PAM images to near-full-resolution quality, significantly outperforming conventional methods in both quantitative metrics and visual fidelity on leaf vein and in vivo mouse ear and eye data.

ABSTRACT

Photoacoustic microscopy (PAM) has been a promising biomedical imaging technology in recent years. However, the point-by-point scanning mechanism results in low-speed imaging, which limits the application of PAM. Reducing sampling density can naturally shorten image acquisition time, which is at the cost of image quality. In this work, we propose a method using convolutional neural networks (CNNs) to improve the quality of sparse PAM images, thereby speeding up image acquisition while keeping good image quality. The CNN model utilizes both squeeze-and-excitation blocks and residual blocks to achieve the enhancement, which is a mapping from a 1/4 or 1/16 low-sampling sparse PAM image to a latent fully-sampled image. The perceptual loss function is applied to keep the fidelity of images. The model is mainly trained and validated on PAM images of leaf veins. The experiments show the effectiveness of our proposed method, which significantly outperforms existing methods quantitatively and qualitatively. Our model is also tested using in vivo PAM images of blood vessels of mouse ears and eyes. The results show that the model can enhance the image quality of the sparse PAM image of blood vessels from several aspects, which may help fast PAM and facilitate its clinical applications.

Motivation & Objective

• To accelerate photoacoustic microscopy (PAM) imaging by reducing scanning points through sparse data acquisition.
• To address the trade-off between imaging speed and image quality in point-by-point scanned PAM systems.
• To develop a deep learning-based image reconstruction method that restores high-fidelity PAM images from low-sampling data.
• To validate the method on both ex vivo leaf vein samples and in vivo mouse tissue (ear and eye) for broad biomedical applicability.
• To demonstrate the robustness of the CNN model to different tissue patterns, including tree-like and radial vascular structures.

Proposed method

• A custom CNN architecture combines residual blocks for feature learning and squeeze-and-excitation (SE) blocks to enhance channel-wise feature importance.
• The model performs image super-resolution by mapping low-sampling PAM images (1/4 or 1/16 of full resolution) to a latent fully-sampled image.
• A perceptual loss function is used during training to preserve fine textures and structural details, avoiding the blurring common in MSE-only training.
• The network is trained and validated on a dataset of 268 full-sampling PAM images of oak and magnolia leaf veins, with low-sampling versions generated via uniform spatial downsampling.
• The model is evaluated using PSNR and SSIM metrics and tested on in vivo PAM data without transfer learning, demonstrating zero-shot generalization to biological tissues.
• Image reconstruction is performed using a 2D maximum amplitude projection (MAP) of the PAM data along the depth direction, which is used as input and output for the network.

Experimental results

Research questions

• RQ1 Can a deep CNN effectively restore high-quality PAM images from sparse data with only 1/4 or 1/16 of the original sampling points?
• RQ2 How does the inclusion of squeeze-and-excitation blocks and perceptual loss improve image reconstruction over standard CNNs with MSE loss?
• RQ3 Can a model trained on leaf vein patterns generalize to in vivo PAM images of mouse ear and eye with different vascular morphologies?
• RQ4 To what extent does the proposed method improve imaging speed while maintaining diagnostic image quality in PAM?
• RQ5 Does the perceptual loss function lead to better preservation of fine anatomical structures compared to pixel-wise MSE loss?

Key findings

• The proposed CNN model significantly outperforms bicubic interpolation and EDSR in both PSNR and SSIM on 1/4 and 1/16 sampling cases, with improvements of up to 1.3876 dB in PSNR and 0.0936 in SSIM when SE blocks are included.
• The perceptual loss function leads to visually superior reconstructions, preserving fine textures and small branches—such as those indicated by blue arrows in Fig. 4—that are lost in MSE-trained models.
• On in vivo mouse ear PAM images with 4× scaling, the model achieves a PSNR of 28.7 dB and SSIM of 0.89, outperforming baseline methods in both quantitative and qualitative assessments.
• For mouse eye images with radial vascular patterns, the model maintains high structural fidelity and edge sharpness, demonstrating robustness to pattern variations beyond the training data.
• The model achieves a 4× speedup in image acquisition (1/16 sampling) while restoring image quality comparable to full-sampling PAM, enabling faster dynamic imaging.
• Ablation studies confirm that SE blocks contribute significantly to performance, improving PSNR and SSIM by over 1 dB and 0.09, respectively, under 4× scaling.