[Paper Review] Deep Learning for Ultrasound Beamforming

Ruud J. G. van Sloun, Jong Chul Ye+1 more|arXiv (Cornell University)|Sep 23, 2021

Ultrasound Imaging and Elastography4 citations

TL;DR

This paper proposes a switchable deep beamformer using adaptive instance normalization (AdaIN) layers to unify multiple ultrasound beamforming methods—such as DAS, MVBF, and deconvolution—into a single neural network. By conditioning the network with learnable AdaIN codes, it generates diverse beamforming outputs without retraining, significantly reducing memory overhead while improving image quality over conventional methods.

ABSTRACT

Diagnostic imaging plays a critical role in healthcare, serving as a fundamental asset for timely diagnosis, disease staging and management as well as for treatment choice, planning, guidance, and follow-up. Among the diagnostic imaging options, ultrasound imaging is uniquely positioned, being a highly cost-effective modality that offers the clinician an unmatched and invaluable level of interaction, enabled by its real-time nature. Ultrasound probes are becoming increasingly compact and portable, with the market demand for low-cost pocket-sized and (in-body) miniaturized devices expanding. At the same time, there is a strong trend towards 3D imaging and the use of high-frame-rate imaging schemes; both accompanied by dramatically increasing data rates that pose a heavy burden on the probe-system communication and subsequent image reconstruction algorithms. With the demand for high-quality image reconstruction and signal extraction from less (e.g unfocused or parallel) transmissions that facilitate fast imaging, and a push towards compact probes, modern ultrasound imaging leans heavily on innovations in powerful digital receive channel processing. Beamforming, the process of mapping received ultrasound echoes to the spatial image domain, naturally lies at the heart of the ultrasound image formation chain. In this chapter on Deep Learning for Ultrasound Beamforming, we discuss why and when deep learning methods can play a compelling role in the digital beamforming pipeline, and then show how these data-driven systems can be leveraged for improved ultrasound image reconstruction.

Motivation & Objective

• To address the high memory and computational cost of training separate deep learning models for each beamforming method (e.g., DAS, MVBF) in clinical ultrasound systems.
• To enable a single deep learning model to generate multiple beamforming outputs (e.g., DAS, MVBF, despeckled images) using conditional AdaIN layers.
• To reduce the need for paired training data by exploring unsupervised learning strategies such as CycleGAN for image quality enhancement.
• To improve ultrasound image fidelity, resolution, and contrast using data-driven beamforming while maintaining real-time performance.

Proposed method

• A deep neural network is trained to map raw ultrasound channel data to beamformed images using supervised learning with paired input-target data.
• The network incorporates adaptive instance normalization (AdaIN) layers that condition feature maps using style codes derived from target beamforming methods.
• The AdaIN transformation reweights the mean and variance of feature maps to match those of a reference style (e.g., DAS or MVBF), enabling dynamic output switching.
• The AdaIN codes are generated by a lightweight network, allowing efficient inference without storing multiple full models.
• For unsupervised learning, CycleGAN-based unpaired image translation is used to improve image quality from low-end systems using high-quality images from different sources.
• The system supports switchable beamforming by changing only the AdaIN code, enabling real-time adaptation to different imaging settings.

Figure 1: An illustration of three transmit types. Transmit events are denoted by $e$ , receive channels as $c$ and, for focused scanlines, $2L+1$ is the size of the active aperture in terms of elements.

Experimental results

Research questions

• RQ1 Can a single deep neural network be trained to emulate multiple beamforming techniques (e.g., DAS, MVBF) without separate models for each?
• RQ2 How effectively can AdaIN layers be used to condition a deep beamformer to generate diverse beamforming outputs?
• RQ3 What is the impact of unsupervised learning on beamforming quality when ground-truth paired data is unavailable?
• RQ4 Can a unified beamformer reduce memory and computational costs in clinical ultrasound systems while maintaining or improving image quality?

Key findings

• A single deep beamformer with AdaIN layers successfully generates multiple beamforming outputs (e.g., DAS, MVBF, deconvolved, despeckled) by conditioning on different AdaIN codes.
• The AdaIN-based approach reduces memory overhead compared to training separate models for each beamforming method, enabling practical deployment in portable ultrasound systems.
• Unsupervised learning via CycleGAN enables image quality improvement using unpaired data, demonstrating feasibility in real-world scenarios where ground-truth data is unavailable.
• The method achieves improved image contrast and resolution over traditional beamforming techniques, particularly in low-SNR and low-frame-rate conditions.

Figure 2: General beamforming scheme.