[Paper Review] A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective

Sin Kit Lo, Qinghua Lu+3 more|arXiv (Cornell University)|Jul 22, 2020

Privacy-Preserving Technologies in Data212 references36 citations

TL;DR

A systematic literature review analyzing federated learning from a software engineering lens, synthesizing 231 primary studies across the federated learning lifecycle (background, requirements, architecture, implementation, evaluation) and identifying trends and challenges.

ABSTRACT

Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. To identify the state-of-the-art in federated learning and explore how to develop federated learning systems, we perform a systematic literature review from a software engineering perspective, based on 231 primary studies. Our data synthesis covers the lifecycle of federated learning system development that includes background understanding, requirement analysis, architecture design, implementation, and evaluation. We highlight and summarise the findings from the results, and identify future trends to encourage researchers to advance their current work.

Motivation & Objective

• Provide a holistic overview of federated learning system development from a software engineering viewpoint.
• Identify the research activities, topics, and experts in federated learning.
• Analyze federated learning requirements, architecture design, implementation, and evaluation across SDLC stages.
• Highlight challenges and future trends to guide practitioners and researchers.
• Offer a data-driven synthesis to inform adoption and system development decisions.

Proposed method

• Systematic literature review following Kitchenham’s guideline (2007).
• Source selection from ACM DL, IEEE Xplore, ScienceDirect, Springer, ArXiv, and Google Scholar (2016-2020).
• Inclusion/exclusion criteria and quality assessment with four quality criteria (citation rate, methodology, findings, future work).
• Data extraction of 231 primary studies with cross-checked coding by two researchers.
• Forward and backward snowballing to augment the initial set.
• Classification of studies along the federated learning software lifecycle (background, requirements, architecture, implementation, evaluation).

Experimental results

Research questions

• RQ1 RQ1: What is federated learning?
• RQ2 RQ1.1: Why is federated learning adopted?
• RQ3 RQ1.2: What are the applications of federated learning?
• RQ4 RQ1.3: What data does the federated learning applications deal with?
• RQ5 RQ1.4: What is the client data distribution of the federated learning applications?

Key findings

• Federated learning is a distributed/central-coordinated or decentralized training approach where local data never leaves devices; updates or gradients are shared to form a global model.
• Motivations for adoption center on data privacy and communication efficiency, with scalability benefits; model performance under non-IID data remains a challenge.
• Applications and data types show image data, structured data, and text are most common; MNIST is a frequently used dataset; many studies are proofs-of-concept or simulations rather than production systems.
• Client data distributions are frequently non-IID, IID, or not specified; non-IID distributions significantly influence model performance and are a focus of FedAvg extensions.
• Research activity on communication efficiency, heterogeneity (statistical and system), data security, and security of clients has surged, especially around 2019; challenges include dropout handling, robustness to non-IID, and scalability.