[Paper Review] Resource-Constrained Decentralized Federated Learning via Personalized Event-Triggering

Shahryar Zehtabi, Seyyedali Hosseinalipour+1 more|arXiv (Cornell University)|Nov 23, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a resource-constrained, decentralized federated learning framework using personalized event-triggered communication to reduce redundant transmissions and improve efficiency in edge devices. By dynamically triggering model updates based on local parameter changes and available resources, the method achieves convergence to the global optimal model at a rate of $\mathcal{O}(\frac{\ln k}{\sqrt{k}})$, outperforming existing decentralized FL baselines in convergence speed and communication efficiency.

ABSTRACT

Federated learning (FL) is a popular technique for distributing machine learning (ML) across a set of edge devices. In this paper, we study fully decentralized FL, where in addition to devices conducting training locally, they carry out model aggregations via cooperative consensus formation over device-to-device (D2D) networks. We introduce asynchronous, event-triggered communications among the devices to handle settings where access to a central server is not feasible. To account for the inherent resource heterogeneity and statistical diversity challenges in FL, we define personalized communication triggering conditions at each device that weigh the change in local model parameters against the available local network resources. We theoretically recover the $O(\ln{k} / \sqrt{k})$ convergence rate to the globally optimal model of decentralized gradient descent (DGD) methods in the setup of our methodology. We provide our convergence guarantees for the last iterates of models, under relaxed graph connectivity and data heterogeneity assumptions compared with the existing literature. To do so, we demonstrate a $B$-connected information flow guarantee in the presence of sporadic communications over the time-varying D2D graph. Our subsequent numerical evaluations demonstrate that our methodology obtains substantial improvements in convergence speed and/or communication savings compared to existing decentralized FL baselines.

Motivation & Objective

• To address the limitations of centralized federated learning in ad hoc and resource-constrained edge networks where a central coordinator is unavailable or impractical.
• To eliminate timing coordination and reduce redundant communication in decentralized FL by introducing asynchronous, event-triggered model aggregation.
• To account for device-level resource heterogeneity by defining personalized communication triggering conditions based on local model changes and available resources.
• To theoretically establish convergence to the globally optimal model under standard distributed learning assumptions.
• To empirically validate improved convergence speed and communication savings compared to existing decentralized FL baselines.

Proposed method

• Devices perform local model training and communicate only when a personalized event-triggering condition is met, based on changes in local model parameters and local resource availability.
• A decentralized consensus mechanism is employed using one-hop neighbor communication over a physical network graph, replacing the star topology of centralized FL.
• The communication triggering condition is defined as a threshold that balances model update significance against local energy and bandwidth constraints.
• A diminishing step size policy $\alpha^{(k)} = \frac{\alpha^{(0)}}{(1 + \frac{k}{\gamma})^\theta}$ is used, with $\theta \in (0.5, 1]$, to ensure convergence.
• Theoretical analysis uses logarithmic integral functions and bounds on consensus error to derive convergence rates under standard assumptions in distributed optimization.
• Numerical evaluations use LeNet5 on Fashion-MNIST with non-i.i.d. data and random geometric communication topologies to validate performance.

Experimental results

Research questions

• RQ1 Can a fully decentralized federated learning system be designed without relying on a central coordinator or timing synchronization?
• RQ2 How can event-triggered communication be personalized per device to account for heterogeneous resource constraints in edge devices?
• RQ3 What is the theoretical convergence rate of a decentralized FL system using personalized event-triggered communication under standard assumptions?
• RQ4 Does the proposed method maintain performance gains in non-convex learning settings such as deep neural networks?
• RQ5 How does the proposed method compare to existing decentralized FL baselines in terms of convergence speed and communication efficiency?

Key findings

• The proposed method achieves a global convergence rate of $\mathcal{O}(\frac{\ln k}{\sqrt{k}})$ under standard assumptions in distributed learning and consensus literature.
• The method significantly reduces communication overhead by triggering transmissions only when meaningful model changes occur, based on personalized thresholds.
• Numerical evaluations on LeNet5 with Fashion-MNIST show that the method maintains strong performance even without the convexity assumption, confirming robustness in non-convex settings.
• The approach outperforms existing decentralized FL baselines in both convergence speed and communication efficiency, particularly in heterogeneous and resource-constrained environments.
• The theoretical analysis confirms that the use of a diminishing step size policy with $\theta \in (0.5, 1]$ ensures convergence, with the rate depending on the choice of $\theta$.
• The logarithmic integral function is used to bound the consensus error, enabling tight theoretical convergence analysis under asynchronous, event-triggered communication.