[Paper Review] Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis

Hong Xing, Osvaldo Simeone+1 more|arXiv (Cornell University)|Jan 29, 2021

Privacy-Preserving Technologies in DataComputer Science55 references114 citations

TL;DR

This paper proposes digital and analog wireless implementations of decentralized stochastic gradient descent (DSGD) for federated learning over device-to-device (D2D) networks, leveraging random linear coding (RLC) and over-the-air computation (AirComp). It establishes convergence bounds showing the optimality gap depends on network connectivity and signal-to-noise ratio (SNR), validated experimentally on image classification with theoretical guarantees under convexity and connectivity assumptions.

ABSTRACT

The proliferation of Internet-of-Things (IoT) devices and cloud-computing applications over siloed data centers is motivating renewed interest in the collaborative training of a shared model by multiple individual clients via federated learning (FL). To improve the communication efficiency of FL implementations in wireless systems, recent works have proposed compression and dimension reduction mechanisms, along with digital and analog transmission schemes that account for channel noise, fading, and interference. The prior art has mainly focused on star topologies consisting of distributed clients and a central server. In contrast, this paper studies FL over wireless device-to-device (D2D) networks by providing theoretical insights into the performance of digital and analog implementations of decentralized stochastic gradient descent (DSGD). First, we introduce generic digital and analog wireless implementations of communication-efficient DSGD algorithms, leveraging random linear coding (RLC) for compression and over-the-air computation (AirComp) for simultaneous analog transmissions. Next, under the assumptions of convexity and connectivity, we provide convergence bounds for both implementations. The results demonstrate the dependence of the optimality gap on the connectivity and on the signal-to-noise ratio (SNR) levels in the network. The analysis is corroborated by experiments on an image-classification task.

Motivation & Objective

• To address the lack of theoretical convergence analysis for wireless D2D federated learning under realistic channel impairments such as noise, fading, and interference.
• To develop communication-efficient DSGD algorithms for decentralized FL in wireless D2D networks using digital (RLC-based compression) and analog (AirComp) transmission.
• To analyze the convergence of these implementations under convex and connected network assumptions, quantifying the impact of SNR and network connectivity on the optimality gap.
• To provide theoretical performance bounds that account for physical layer impairments and general scheduling/power allocation policies, extending prior work limited to ideal or noiseless channels.
• To validate the theoretical findings through experiments on an image classification task, demonstrating the practical viability of the proposed wireless DSGD frameworks.

Proposed method

• Proposes a digital wireless DSGD implementation using random linear coding (RLC) for gradient compression, reducing communication overhead while preserving model accuracy.
• Introduces an analog wireless DSGD implementation based on over-the-air computation (AirComp), enabling simultaneous analog transmission of gradients over the wireless medium via waveform superposition.
• Derives convergence bounds for both digital and analog implementations under convex objectives, using a Lyapunov-style analysis to track consensus error and optimality gap.
• Introduces a novel analysis framework that models the impact of channel noise, fading, and interference through signal-to-noise ratio (SNR) and network connectivity parameters.
• Employs a modified CHOCO-SGD algorithm as the baseline, incorporating compression and consensus mechanisms, with convergence analysis extended to wireless settings.
• Uses a time-varying learning rate policy and incorporates error feedback mechanisms to bound the effect of compression and channel noise on convergence.

Experimental results

Research questions

• RQ1 How does wireless channel noise and fading affect the convergence of decentralized stochastic gradient descent (DSGD) in device-to-device (D2D) federated learning networks?
• RQ2 What is the theoretical impact of network connectivity and signal-to-noise ratio (SNR) on the optimality gap in wireless DSGD implementations?
• RQ3 Can over-the-air computation (AirComp) enable efficient, simultaneous gradient aggregation in D2D FL without decoding individual transmissions?
• RQ4 How do compression techniques like random linear coding (RLC) affect convergence in wireless D2D FL under practical physical layer constraints?
• RQ5 What are the performance limits of analog versus digital wireless DSGD in terms of convergence rate and robustness to channel impairments?

Key findings

• The convergence of the proposed digital and analog DSGD implementations is theoretically bounded, with the optimality gap inversely proportional to network connectivity and SNR.
• The analysis shows that the optimality gap is upper bounded by a term that depends on the inverse of the minimum eigenvalue of the network’s Laplacian matrix, reflecting the impact of connectivity.
• For the analog implementation via AirComp, the convergence is limited by a noise-induced error floor, which is quantified and shown to be controllable under proper power and scheduling policies.
• The digital RLC-based implementation achieves a convergence rate that scales with the square of the learning rate and the compression error, with a bound that depends on the SNR and network topology.
• Experimental results on an image classification task confirm the theoretical predictions, showing stable convergence under varying SNR and connectivity levels.
• The derived convergence bounds are robust to general scheduling and power allocation policies, making the framework applicable to real-world wireless D2D networks.