[Paper Review] Federated Learning: Opportunities and Challenges

Priyanka Mary Mammen|arXiv (Cornell University)|Jan 14, 2021

Privacy-Preserving Technologies in DataComputer Science35 references144 citations

TL;DR

Federated Learning enables collaborative ML without sharing raw data, but faces training, privacy, and security challenges; the paper surveys architectures, applications, defenses, and recent developments.

ABSTRACT

Federated Learning (FL) is a concept first introduced by Google in 2016, in which multiple devices collaboratively learn a machine learning model without sharing their private data under the supervision of a central server. This offers ample opportunities in critical domains such as healthcare, finance etc, where it is risky to share private user information to other organisations or devices. While FL appears to be a promising Machine Learning (ML) technique to keep the local data private, it is also vulnerable to attacks like other ML models. Given the growing interest in the FL domain, this report discusses the opportunities and challenges in federated learning.

Motivation & Objective

• Motivate Federated Learning as a privacy-preserving collaborative learning paradigm for sensitive domains.
• Provide a structured overview of FL architectures, types, and applications.
• Identify training bottlenecks and heterogeneity issues in FL systems.
• Survey privacy and security threats and defenses in FL.
• Highlight recent developments and future research directions in FL.

Proposed method

• Literature survey of key FL architectures and categories (vertical, horizontal, transfer, cross-silo, cross-device).
• Discussion of training bottlenecks such as communication overhead and data/systems heterogeneity (FedAvg and alternatives).
• Analysis of privacy and security threats including membership inference, data poisoning, model poisoning, backdoor attacks, and defenses (secure computation, differential privacy, trusted execution environments).
• Review of recent FL developments (one-shot FL, incentive mechanisms, FL as a service, asynchronous FL, blockchain-based FL).
• Illustration of FL applications in healthcare, transportation, finance, and NLP.

Experimental results

Research questions

• RQ1 What are the main FL architectures and how do they differ in data distribution and participation?
• RQ2 What are the primary training, privacy, and security challenges in FL, and what defenses exist or are proposed?
• RQ3 What are the key application domains where FL is beneficial, and what examples demonstrate its potential?
• RQ4 What are the emerging developments and directions that can address FL's scalability, incentives, and privacy concerns?

Key findings

• Federated Learning enables collaborative model training without exchanging private data, with potential across healthcare, transportation, finance, and NLP.
• Training in FL is challenged by communication overhead and heterogeneity of devices and data, requiring new aggregation and efficiency techniques.
• FL is vulnerable to a range of attacks, including membership inference, data poisoning, model poisoning, and backdoor attacks, prompting defense mechanisms like secure computation, differential privacy, and trusted execution environments.
• Defenses often reduce model accuracy or impose computational overhead; balancing privacy, security, and performance remains an open problem.
• Recent developments include one-shot and distilled one-shot FL, incentive mechanisms, FL as a service, asynchronous FL, and blockchain-based FL frameworks.