[Paper Review] Limited-view Photoacoustic Imaging Reconstruction With Dual Domain Inputs Under Mutual Information Constraint

Jiadong Zhang, Hengrong Lan+4 more|arXiv (Cornell University)|Nov 12, 2020

Photoacoustic and Ultrasonic Imaging14 references4 citations

TL;DR

This paper proposes DuDoUnet, a dual-domain U-Net with an Information Sharing Block (ISB) that leverages both time-domain (DAS) and frequency-domain (k-space) photoacoustic images to improve limited-view reconstruction. By incorporating mutual information regularization via a pre-trained auxiliary network, the model enhances prior knowledge learning, achieving SSIM of 93.56% and PSNR of 20.89 dB on a clinical database, outperforming state-of-the-art methods.

ABSTRACT

Based on photoacoustic effect, photoacoustic tomography is developing very fast in recent years, and becoming an important imaging tool for both preclinical and clinical studies. With enough ultrasound transducers placed around the biological tissue, PAT can provide both deep penetration and high image contrast by hybrid usage of light and sound. However, considering space and measurement environmental limitations, transducers are always placed in a limited-angle way, which means that the other side without transducer coverage suffers severe information loss. With conventional image reconstruction algorithms, the limited-view tissue induces artifacts and information loss, which may cause doctors misdiagnosis or missed diagnosis. In order to solve limited-view PA imaging reconstruction problem, we propose to use both time domain and frequency domain reconstruction algorithms to get delay-and-sum (DAS) image inputs and k-space image inputs. These dual domain images share nearly same texture information but different artifact information, which can teach network how to distinguish these two kinds of information at input level. In this paper, we propose Dual Domain Unet (DuDoUnet) with specially designed Information Sharing Block (ISB), which can further share two domains' information and distinguish artifacts. Besides, we use mutual information (MI) with an auxiliary network, whose inputs and outputs are both ground truth, to compensate prior knowledge of limited-view PA inputs. The proposed method is verified with a public clinical database, and shows superior results with SSIM = 93.5622% and PSNR = 20.8859.

Motivation & Objective

• To address severe image artifacts and information loss in limited-view photoacoustic tomography (PAT) due to incomplete transducer coverage.
• To improve reconstruction quality by leveraging complementary information from time-domain (DAS) and frequency-domain (k-space) image inputs.
• To enhance deep learning-based reconstruction by enabling the network to distinguish texture from artifacts at the input level through dual-domain feature fusion.
• To compensate for limited-view induced information loss by incorporating mutual information as an auxiliary constraint to improve prior knowledge learning.

Proposed method

• The proposed DuDoUnet uses an Information Sharing Block (ISB) that processes both DAS and k-space images in parallel, extracting low-level raw features and high-level features from both domains.
• ISB employs FFT and IFFT to transform DAS and k-space inputs, enabling cross-domain feature sharing while preserving domain-specific characteristics.
• The network fuses multi-level features from both domains through concatenation and convolutional layers, improving texture and artifact discrimination.
• A variational mutual information constraint is applied via a pre-trained auxiliary encoder-decoder network that takes ground truth as both input and output to learn compact, informative latent representations.
• The mutual information loss is formulated using a conditional Gaussian distribution with learnable mean and variance, stabilizing training via a sigmoid-based variance estimator.
• The overall training objective combines reconstruction loss (MSE) and mutual information regularization, optimizing for both fidelity and prior knowledge.

Experimental results

Research questions

• RQ1 Can dual-domain inputs (time and frequency domain) improve limited-view photoacoustic image reconstruction by providing complementary artifact and texture information?
• RQ2 How does an Information Sharing Block (ISB) enhance feature fusion and artifact discrimination when processing DAS and k-space images?
• RQ3 To what extent does mutual information regularization improve prior knowledge learning in deep learning-based PAT reconstruction?
• RQ4 Does the proposed method outperform existing U-Net and dual-encoder architectures in reconstructing fine vessel structures under limited-view conditions?
• RQ5 Can mutual information constraint stabilize training and improve generalization in low-data or limited-view imaging scenarios?

Key findings

• The proposed DuDoUnet with mutual information constraint achieved a structural similarity index (SSIM) of 93.5622%, significantly outperforming all baselines.
• The method achieved a peak signal-to-noise ratio (PSNR) of 20.8859 dB, demonstrating superior reconstruction fidelity compared to other models.
• DuDoUnet+MI outperformed Unet#1 (SSIM: 90.99, PSNR: 19.40), Unet#2 (SSIM: 91.46, PSNR: 19.77), Ynet (SSIM: 87.25, PSNR: 17.82), and DEUnet (SSIM: 90.65, PSNR: 19.30).
• Qualitative results showed that DuDoUnet+MI successfully recovered fine vessel topologies and preserved structural continuity, while other methods produced fragmented or spurious structures.
• The ablation study confirmed that dual-domain inputs and mutual information regularization both contributed significantly to performance gains.
• The use of FFT and IFFT in ISB enabled effective cross-domain feature learning without increasing model complexity due to domain-specific operations.