[Paper Review] Communication Efficiency in Federated Learning: Achievements and Challenges

Osama Shahid, Seyedamin Pouriyeh+4 more|arXiv (Cornell University)|Jul 23, 2021

Privacy-Preserving Technologies in DataComputer Science76 references50 citations

TL;DR

This survey analyzes communication efficiency in Federated Learning, detailing challenges and methods to reduce communication overhead.

ABSTRACT

Federated Learning (FL) is known to perform Machine Learning tasks in a distributed manner. Over the years, this has become an emerging technology especially with various data protection and privacy policies being imposed FL allows performing machine learning tasks whilst adhering to these challenges. As with the emerging of any new technology, there are going to be challenges and benefits. A challenge that exists in FL is the communication costs, as FL takes place in a distributed environment where devices connected over the network have to constantly share their updates this can create a communication bottleneck. In this paper, we present a survey of the research that is performed to overcome the communication constraints in an FL setting.

Motivation & Objective

• Explain the motivation for Federated Learning and its privacy-preserving benefits.
• Identify and categorize the communication-related challenges in FL.
• Review and synthesize methods to improve communication efficiency in FL.
• Highlight gaps and future directions for research on FL communication.
• Position this work as a focused survey on FL communication rather than general FL topics.

Proposed method

• Describe the FL architecture with data owners and central server.
• Introduce types of FL systems (horizontal, vertical, federated transfer).
• Formulate two research questions focusing on communication challenges and efficiency methods.
• Review literature on local updating, client selection, model update reduction, decentralized training, and compression.
• Discuss existing surveys and position this work as a dedicated communication-focused survey.

Experimental results

Research questions

• RQ1 What are the key challenges in Federated Learning related to communication?
• RQ2 What methods can make communication more efficient in Federated Learning?

Key findings

• Identification of main communication bottlenecks: number of devices, network bandwidth, edge computation limits, and data heterogeneity.
• Local updating, client selection, and model update reduction can reduce communication rounds and data exchanged.
• FedAvg, SCAFFOLD, FedDANE, and FedPAQ offer strategies to mitigate drift and overhead.
• Participation strategies like FedCS and MCML enhance scheduling under constrained networks.
• One-shot and ensemble approaches provide alternatives to frequent round-based updates.
• Decentralized and peer-to-peer learning with compression contribute to communication efficiency.