[Paper Review] Bayesian Federated Learning: A Survey

Longbing Cao, Hui Chen+4 more|arXiv (Cornell University)|Apr 26, 2023

Privacy-Preserving Technologies in Data7 citations

TL;DR

This survey introduces Bayesian Federated Learning (BFL) as a robust framework that integrates Bayesian learning with federated learning to address challenges like data scarcity, model uncertainty, and system heterogeneity. By modeling distributions over model parameters and leveraging prior knowledge, BFL enables more reliable, explainable, and dynamic predictions in privacy-sensitive, real-world applications.

ABSTRACT

Federated learning (FL) demonstrates its advantages in integrating distributed infrastructure, communication, computing and learning in a privacy-preserving manner. However, the robustness and capabilities of existing FL methods are challenged by limited and dynamic data and conditions, complexities including heterogeneities and uncertainties, and analytical explainability. Bayesian federated learning (BFL) has emerged as a promising approach to address these issues. This survey presents a critical overview of BFL, including its basic concepts, its relations to Bayesian learning in the context of FL, and a taxonomy of BFL from both Bayesian and federated perspectives. We categorize and discuss client- and server-side and FL-based BFL methods and their pros and cons. The limitations of the existing BFL methods and the future directions of BFL research further address the intricate requirements of real-life FL applications.

Motivation & Objective

• To address the limitations of conventional federated learning in handling limited, dynamic, and non-IID data with high uncertainty.
• To integrate Bayesian learning's strengths—prior knowledge, uncertainty quantification, and robustness—into federated learning frameworks.
• To provide a systematic taxonomy of BFL methods from both Bayesian and federated learning perspectives.
• To identify key technical gaps and future research directions for actionable, real-world BFL applications.
• To support the development of explainable, fair, and secure decentralized AI systems through uncertainty-aware learning.

Proposed method

• Proposes a unified framework that combines Bayesian learning with federated learning to model distributions over model parameters and quantify uncertainty.
• Classifies BFL methods into client-side and server-side approaches, and further categorizes them based on FL-specific challenges such as communication efficiency and heterogeneity.
• Introduces a dual taxonomy combining Bayesian learning techniques (e.g., variational inference, Monte Carlo dropout) and federated learning concerns (e.g., non-IID data, dynamic clients).
• Analyzes BFL methods through the lens of both Bayesian principles (prior specification, posterior inference) and federated system design (aggregation, communication, privacy).
• Examines hybrid BFL architectures that combine Bayesian methods with other paradigms, such as transfer learning, reinforcement learning, and edge computing.
• Proposes evaluation criteria that include not only technical metrics but also domain-driven, ethical, and actionable performance indicators like fairness, explainability, and verifiability.

Experimental results

Research questions

• RQ1 How can Bayesian learning enhance the robustness and uncertainty quantification of federated learning under data scarcity and non-IID conditions?
• RQ2 What are the key differences and trade-offs between client-side and server-side Bayesian federated learning methods in terms of scalability, communication, and accuracy?
• RQ3 How can BFL effectively handle complex data characteristics such as concept drift, label scarcity, and mixed modalities?
• RQ4 What are the major technical gaps in current BFL approaches that hinder their deployment in real-world, production-grade decentralized AI systems?
• RQ5 How can BFL be extended to support hybrid, multi-source, and multi-modal learning scenarios while maintaining privacy and efficiency?

Key findings

• BFL significantly improves model robustness and uncertainty calibration in low-data and dynamic client environments by leveraging prior knowledge and posterior inference.
• Client-side BFL methods offer better personalization and privacy but face challenges in communication efficiency and convergence stability.
• Server-side BFL enables centralized uncertainty modeling and global consistency but risks overfitting to dominant clients and lacks fine-grained personalization.
• Hybrid BFL architectures—such as those combining Bayesian methods with transfer learning, edge inference, or compressed communication—show promise in scaling to complex, real-world deployments.
• Current BFL methods still struggle with computational and communication overhead, especially in asynchronous, decentralized, or resource-constrained settings.
• Actionable BFL requires not only technical improvements but also integration of ethical, explainable, and verifiable design principles to meet real-world deployment demands.