[Paper Review] A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Mingzhe Chen, Zhaohui Yang+4 more|arXiv (Cornell University)|Sep 17, 2019

Privacy-Preserving Technologies in DataComputer Science39 references107 citations

TL;DR

The paper presents a joint learning and wireless framework for federated learning over wireless networks, deriving convergence impact of wireless factors and optimizing user selection, resource allocation, and transmit power to minimize FL loss.

ABSTRACT

In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that will generate a global FL model and send it back to the users. Since all training parameters are transmitted over wireless links, the quality of the training will be affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS must select an appropriate subset of users to execute the FL algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To address this problem, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can reduce the FL loss function value by up to 10% and 16%, respectively, compared to: 1) An optimal user selection algorithm with random resource allocation and 2) a standard FL algorithm with random user selection and resource allocation.

Motivation & Objective

• Motivate federated learning (FL) over realistic wireless networks where local models are uploaded to a base station to form a global model.
• quantify how wireless factors (packet errors, RB allocation, power) affect FL convergence and performance.
• Develop an optimization framework that jointly optimizes user selection, uplink RB allocation, and transmit power to minimize FL loss.
• Provide closed-form convergence insights to guide resource allocation and learning rate settings.
• Demonstrate performance gains via simulations against baseline FL and wireless-only optimization approaches.

Proposed method

• Model the FL process with a BS aggregating local models from selected users, forming a global model transmitted back to users.
• Derive a closed-form expression for the expected FL convergence rate that links packet error rates to FL performance.
• Formulate a mixed-integer nonlinear program to minimize FL loss subject to delay, energy, and resource constraints.
• Compute optimal transmit powers under given user selection and RB allocation, then transform the problem into a bipartite matching problem solved by the Hungarian algorithm for FL-aware user selection and RB allocation.
• Analyze how transmit power, RB allocation, and user participation affect convergence and provide guidelines for learning rate and participation level.

Experimental results

Research questions

• RQ1 How do wireless transmission errors and resource constraints impact the convergence and accuracy of federated learning over wireless networks?
• RQ2 What is the optimal combination of user selection, uplink RB allocation, and transmit power to minimize FL loss while satisfying delay and energy constraints?
• RQ3 Can the FL process be effectively optimized via a closed-form convergence rate to guide resource allocation decisions?
• RQ4 How do learning rate and participation level influence FL robustness to wireless-induced errors?

Key findings

• The proposed joint framework can improve identification accuracy by up to 1.4%, 3.5%, and 4.1% under specific baselines.
• Deriving the FL convergence bound shows the packet error rate, RB allocation, and user selection significantly affect convergence speed and final performance.
• Optimal power control and RB allocation, combined with FL-aware user selection, yield gains over random resource allocation and FL without FL-aware scheduling.
• The convergence gap reduces as packet error rates decrease and more users participate, indicating the importance of FL-aware wireless optimization.
• Guidelines are implied: adjusting learning rate and user participation can mitigate wireless error effects and guarantee convergence.