[Paper Review] A Communication Efficient Vertical Federated Learning Framework.

Yang Liu, Yan Kang+6 more|arXiv (Cornell University)|Dec 24, 2019

Privacy-Preserving Technologies in Data59 citations

TL;DR

This paper proposes FedBCD, a communication-efficient vertical federated learning framework that reduces training rounds by leveraging stochastic block coordinate descent. By optimizing batch size, sample size, and local iterations, FedBCD achieves O(√T) communication rounds for O(1/√T) convergence accuracy, significantly cutting communication overhead in cross-organizational AI training.

ABSTRACT

One critical challenge for applying today's Artificial Intelligence (AI) technologies to real-world applications is the common existence of data silos across different organizations. Due to legal, privacy and other practical constraints, data from different organizations cannot be easily integrated. Federated learning (FL), especially the vertical FL (VFL), allows multiple parties having different sets of attributes about the same user collaboratively build models while preserving user privacy. However, communication overhead is a principal bottleneck since the existing VFL protocols require per-iteration communications among all parties. In this paper, we propose the Federated Stochastic Block Coordinate Descent (FedBCD) to effectively reduce the communication rounds for VFL. We show that when the batch size, sample size, and the local iterations are selected appropriately, the algorithm requires $\mathcal{O}(\sqrt{T})$ communication rounds to achieve $\mathcal{O}(1/\sqrt{T})$ accuracy. Finally, we demonstrate the performance of FedBCD on several models and datasets, and on a large-scale industrial platform for VFL.

Motivation & Objective

• To address the high communication overhead in vertical federated learning (VFL), where all parties must communicate in every training iteration.
• To reduce the number of communication rounds in VFL without compromising model accuracy.
• To design a scalable and privacy-preserving training framework suitable for real-world data silos across organizations.
• To validate the theoretical communication efficiency gains on real datasets and industrial-scale platforms.

Proposed method

• Proposes Federated Stochastic Block Coordinate Descent (FedBCD), a novel optimization framework tailored for vertical FL.
• Introduces block coordinate descent updates where only a subset of model parameters is updated per round, reducing per-iteration communication.
• Employs adaptive batching and local iteration scheduling to balance convergence speed and communication cost.
• Derives theoretical convergence bounds showing O(√T) communication rounds for O(1/√T) accuracy under optimal hyperparameter settings.
• Applies the method to various models and datasets, including large-scale industrial deployment.
• Uses secure aggregation and cryptographic techniques to preserve user privacy during cross-party communication.

Experimental results

Research questions

• RQ1 Can communication rounds in vertical federated learning be reduced below the standard O(T) scaling without sacrificing convergence accuracy?
• RQ2 What hyperparameter configurations (batch size, sample size, local iterations) yield optimal communication-accuracy trade-offs in VFL?
• RQ3 Does the proposed FedBCD framework maintain privacy while enabling efficient model training across data silos?
• RQ4 How does FedBCD perform in practice on real-world datasets and industrial-scale platforms?
• RQ5 Can theoretical communication complexity bounds be achieved in real-world VFL deployments?

Key findings

• FedBCD reduces communication rounds to O(√T) for achieving O(1/√T) convergence accuracy, significantly improving efficiency over standard VFL.
• Optimal selection of batch size, sample size, and local iterations enables the theoretical communication complexity to be realized in practice.
• The framework maintains strong model performance across multiple models and datasets, including complex industrial-scale applications.
• Empirical results confirm that FedBCD outperforms baseline VFL methods in communication efficiency while preserving model accuracy.
• The method scales effectively on a large-scale industrial platform, demonstrating real-world viability for cross-organizational AI training.