[Paper Review] Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging

Ziyuan Yang, Wenjun Xia+4 more|arXiv (Cornell University)|Jun 8, 2022

Medical Imaging Techniques and Applications48 references11 citations

TL;DR

This paper proposes HyperFed, a hypernetwork-based personalized federated learning framework for multi-institutional CT image reconstruction that decouples global feature learning from institution-specific data adaptation. By using an institution-specific hypernetwork to generate modulating hyperparameters for a global-sharing imaging network, HyperFed achieves state-of-the-art reconstruction quality while preserving data privacy and effectively mitigating non-iid data distribution across institutions.

ABSTRACT

Computed tomography (CT) is of great importance in clinical practice due to its powerful ability to provide patients' anatomical information without any invasive inspection, but its potential radiation risk is raising people's concerns. Deep learning-based methods are considered promising in CT reconstruction, but these network models are usually trained with the measured data obtained from specific scanning protocol and need to centralizedly collect large amounts of data, which will lead to serious data domain shift, and privacy concerns. To relieve these problems, in this paper, we propose a hypernetwork-based federated learning method for personalized CT imaging, dubbed as HyperFed. The basic assumption of HyperFed is that the optimization problem for each institution can be divided into two parts: the local data adaption problem and the global CT imaging problem, which are implemented by an institution-specific hypernetwork and a global-sharing imaging network, respectively. The purpose of global-sharing imaging network is to learn stable and effective common features from different institutions. The institution-specific hypernetwork is carefully designed to obtain hyperparameters to condition the global-sharing imaging network for personalized local CT reconstruction. Experiments show that HyperFed achieves competitive performance in CT reconstruction compared with several other state-of-the-art methods. It is believed as a promising direction to improve CT imaging quality and achieve personalized demands of different institutions or scanners without privacy data sharing. The codes will be released at https://github.com/Zi-YuanYang/HyperFed.

Motivation & Objective

• To address privacy concerns and data sharing restrictions in multi-institutional CT imaging by avoiding centralized data collection.
• To overcome the non-i.i.d. data distribution problem inherent in CT imaging due to scanner and protocol variations across institutions.
• To enable personalized CT reconstruction that maintains high image quality for each institution without compromising global model stability.
• To develop a federated learning framework that integrates personalization through hypernetworks while preserving data privacy.
• To demonstrate that the proposed method outperforms both centralized and standard federated learning baselines in reconstruction quality and robustness.

Proposed method

• The framework decomposes the optimization into two parts: a global-sharing imaging network for learning universal features and an institution-specific hypernetwork for personalization.
• The institution-specific hypernetwork takes scanning parameters as input and generates hyperparameters to condition the global imaging network for each institution’s unique data distribution.
• The global imaging network is trained via federated averaging using gradients from all institutions, ensuring model stability and generalization.
• The hypernetworks are trained locally per institution and do not participate in global aggregation, preserving privacy and enabling domain-specific adaptation.
• A two-stage training process is employed: first, the global imaging network is pre-trained on a large-scale dataset; second, the hypernetworks are fine-tuned per institution using local data.
• The method leverages RED-CNN as the backbone network for image reconstruction and evaluates performance using PSNR and SSIM metrics.

Experimental results

Research questions

• RQ1 Can a hypernetwork-based federated learning framework effectively balance personalization and model stability in multi-institutional CT image reconstruction?
• RQ2 How does the proposed HyperFed method perform compared to centralized and standard federated learning baselines in terms of image quality and robustness?
• RQ3 To what extent can the institution-specific hypernetwork improve reconstruction quality when local data is limited?
• RQ4 Does the separation of global feature learning and local adaptation reduce the impact of non-i.i.d. data distributions in CT imaging?
• RQ5 Can the framework maintain high performance across institutions with varying scanning protocols and data sizes?

Key findings

• On the large-scale training set, HyperFed achieved a PSNR of 38.44 and SSIM of 0.9520 across all institutions, outperforming FedAvg (36.99 PSNR), FedProx (37.30 PSNR), and the non-FL baseline (37.71 PSNR).
• In institution-specific evaluations, HyperFed achieved PSNR values of 39.17 (Institution #1), 39.07 (Institution #2), and 39.41 (Institution #5), consistently outperforming other methods.
• Even with small local datasets, HyperFed maintained competitive performance, while the non-FL baseline lost fine details due to limited data.
• FL-based methods like FedAvg and FedProx showed improved stability and performance with larger datasets, but still failed to achieve optimal local reconstruction quality.
• The boxplot and profile analyses confirmed that HyperFed maintained consistent and superior performance across all institutions, indicating strong robustness and personalization.
• The proposed method effectively mitigates the non-i.i.d. problem in CT imaging by enabling institution-specific adaptation without sharing raw data or gradients from hypernetworks.