[论文解读] Convergence Time Optimization for Federated Learning over Wireless Networks

Mingzhe Chen, H. Vincent Poor+2 more|arXiv (Cornell University)|Jan 22, 2020

Privacy-Preserving Technologies in Data参考文献 31被引用 33

一句话总结

该论文提出联合设计用户选择、无线资源分配和基于人工神经网络的预测，以加速无线网络上的联邦学习收敛并降低训练损失。

ABSTRACT

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL performance and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that enables users of higher importance to be selected more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time while optimizing the FL performance. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on its global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to enhance its global FL model and improve the FL convergence speed and performance.

研究动机与目标

• 在受限资源块（RBs）的现实无线网络中动机化并建模联邦学习。
• 通过联合用户选择和RB分配来最小化FL收敛时间和训练损失。
• 通过神经网络实现对非传输用户本地模型的预测，以提升全局模型。
• 分析调度、资源和预测如何影响收敛速度与学习性能。

提出的方法

• 在RB限制下，基站（BS）在每次迭代中从一部分用户收集本地模型来建模FL。
• 给定用户选择和RB分配，建立联合优化以最小化迭代时间和训练损失。
• 引入一种概率性用户选择方案，优先考虑对全局模型影响强的本地更新的用户。
• 在固定用户选择后，使用内部点方法在每次迭代中求解RB分配问题。
• 使用人工神经网络（MLP）预测非传输用户的本地模型以增强全局模型。
• 在预测误差检查的前提下，使用传输的本地模型和预测的本地模型的组合来更新全局模型。

实验结果

研究问题

• RQ1 在无线约束下，如何设计用户选择和RB分配以最小化FL收敛时间？
• RQ2 预测非传输用户本地模型对FL收敛性和准确性有何影响？
• RQ3 在所提出的无线框架下，学习方法（全梯度与随机梯度）在收敛中扮演何种角色？
• RQ4 所提出的概率性用户选择在确保所有用户对全局模型有贡献方面有多有效？

主要发现

• 在无线网络上的FL收敛时间可通过所提出的设计降低多达56%。
• 基于ANN的非传输用户模型预测可以提升全局模型质量和收敛速度。
• 概率性用户选择提高了包含具有重要影响的本地模型的概率，同时确保每个用户的访问概率非零。
• 通过线性化约束的RB分配实现高效优化和更快的每次迭代更新。
• 在全局更新中包含预测的本地模型可以降低训练损失，并在下游任务（如手写数字识别）上比标准FL提高高达3%的准确性。
• 分析表明全梯度/SGD、RB分配、用户选择和预测准确性对收敛行为具有显著影响。