[论文解读] On the Design of Communication Efficient Federated Learning over Wireless Networks.

Richeng Jin, Xiaofan He+1 more|arXiv (Cornell University)|Apr 15, 2020

Privacy-Preserving Technologies in Data参考文献 18被引用 6

一句话总结

该论文提出了一种面向无线网络的通信高效联邦学习框架，仅传输梯度符号，从而降低开销。通过仅利用接收端信道状态信息（CSI）并优化能量、中断概率与通信轮次之间的权衡，该框架在异构数据和无线网络约束下表现出鲁棒性能，仿真结果验证了其高效性与抗干扰能力。

ABSTRACT

Recently, federated learning (FL), as a promising distributed machine learning approach, has attracted lots of research efforts. In FL, the parameter server and the mobile devices share the training parameters over wireless links. As a result, reducing the communication overhead becomes one of the most critical challenges. Despite that there have been various communication-efficient machine learning algorithms in literature, few of the existing works consider their implementation over wireless networks. In this work, the idea of SignSGD is adopted and only the signs of the gradients are shared between the mobile devices and the parameter server. In addition, different from most of the existing works that consider Channel State Information (CSI) at both the transmitter side and the receiver side, only receiver side CSI is assumed. In such a case, an essential problem for the mobile devices is to select appropriate local processing and communication parameters. In particular, two tradeoffs are observed under a fixed total training time: (i) given the time for each communication round, the energy consumption versus the outage probability per communication round and (ii) given the energy consumption, the number of communication rounds versus the outage probability per communication round. Two optimization problems regarding the aforementioned two tradeoffs are formulated and solved. The first problem minimizes the energy consumption given the outage probability (and therefore the learning performance) requirement while the second problem optimizes the learning performance given the energy consumption requirement. Furthermore, the heterogeneous data distribution scenario is considered and a new algorithm that can deal with heterogeneous data distribution is proposed. Extensive simulations are performed to demonstrate the effectiveness of the proposed method.

研究动机与目标

• 解决无线网络中联邦学习面临的高通信开销挑战。
• 设计一种在无线信道约束下最小化能耗同时保持学习性能的系统。
• 在时间受限的训练环境中，优化通信轮次、能耗与中断概率之间的权衡。
• 应对设备间数据分布异构这一联邦学习中常见的现实挑战。
• 开发一种仅依赖接收端CSI的实用化解决方案，降低系统复杂度。

提出的方法

• 采用SignSGD算法仅传输梯度符号，大幅降低通信量。
• 假设仅使用接收端信道状态信息（CSI），简化系统模型并减少反馈开销。
• 构建两个优化问题：在中断概率约束下最小化能耗，以及在能耗预算下最大化学习性能。
• 提出一种针对异构数据分布的新算法，提升非独立同分布（non-IID）场景下的收敛性与模型准确率。
• 利用时间和能量约束，建模每轮通信持续时间、能耗与中断概率之间的权衡。
• 求解优化问题，推导出移动设备最优的本地处理与通信参数。

实验结果

研究问题

• RQ1 在反馈受限的无线网络联邦学习中，如何最大化通信效率？
• RQ2 在固定训练时间内，能耗与每轮中断概率之间的最优权衡是什么？
• RQ3 当能耗受限时，通信轮次与中断概率之间如何权衡？
• RQ4 仅依赖接收端CSI的通信高效联邦学习框架能否在异构数据分布下保持性能？
• RQ5 哪些参数配置（如轮次持续时间、发射功率）可在满足学习性能（中断）要求的前提下最小化能耗？

主要发现

• 所提方法通过仅传输梯度符号显著降低了通信开销，实现了高效的无线部署。
• 仅依赖接收端CSI，系统降低了反馈复杂度，同时保持了鲁棒性能。
• 优化框架成功平衡了能耗与中断概率，在时间和能耗约束下确保了可靠训练。
• 新算法有效应对了异构数据分布，提升了非独立同分布场景下的模型收敛性与准确率。
• 仿真结果表明，所提方法在能耗更低、中断更少的情况下实现了稳定的性能，优于基线方法。
• 权衡分析表明，延长通信轮次持续时间可提升能量效率，但会增加中断风险，可通过最优参数调优加以缓解。