[Paper Review] Delay Minimization for Federated Learning Over Wireless Communication Networks

Zhaohui Yang, Mingzhe Chen+5 more|arXiv (Cornell University)|Jul 5, 2020

Privacy-Preserving Technologies in Data22 references53 citations

TL;DR

The paper formulates a joint computation-and-communication delay minimization problem for federated learning over wireless networks and solves it via a bisection algorithm, achieving notable delay reductions.

ABSTRACT

In this paper, the problem of delay minimization for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model parameters to a base station (BS) which aggregates the local FL models and broadcasts the aggregated FL model back to all the users. Since FL involves learning model exchanges between the users and the BS, both computation and communication latencies are determined by the required learning accuracy level, which affects the convergence rate of the FL algorithm. This joint learning and communication problem is formulated as a delay minimization problem, where it is proved that the objective function is a convex function of the learning accuracy. Then, a bisection search algorithm is proposed to obtain the optimal solution. Simulation results show that the proposed algorithm can reduce delay by up to 27.3% compared to conventional FL methods.

Motivation & Objective

• Motivate delay-aware FL over wireless networks due to privacy and resource constraints.
• Model local computation and wireless transmission delays influenced by learning accuracy.
• Show that total delay is convex in learning accuracy and derive optimal solutions via bisection.

Proposed method

• Model a cellular FL system with one base station and K users performing local FL with given accuracy η.
• Derive per-user computation time as A_k log2(1/η)/f_k and transmission constraints with FDMA.
• Prove delay is convex in η and apply a bisection search to find optimal η and resource allocation.
• Reduce the joint optimization to a convex problem via Lemmas 4-5 and Theorem 1.
• Propose Algorithm 2 for delay minimization using a bisection over the total training time T.

Experimental results

Research questions

• RQ1 How to minimize the training delay of federated learning over wireless links by jointly optimizing local computation and uplink transmission resources?
• RQ2 Is the FL delay convex with respect to learning accuracy, and can a bisection method yield the optimal solution?
• RQ3 What are the sufficient and necessary conditions for feasible resource allocation under the delay constraint?
• RQ4 How does optimized bandwidth, power, and accuracy trade off to minimize overall delay?

Key findings

• The total FL delay is a convex function of the learning accuracy η.
• A bisection-based algorithm can obtain the optimal delay-minimizing solution.
• Joint optimization of computation capacity, bandwidth allocation, and transmit power yields lower delays than reference schemes.
• Simulation shows the proposed method can reduce delay by up to 27.3% compared to conventional FL methods.