[Paper Review] Hierarchical Federated Learning through LAN-WAN Orchestration

Jinliang Yuan, Mengwei Xu+4 more|arXiv (Cornell University)|Oct 22, 2020

Privacy-Preserving Technologies in Data45 references31 citations

TL;DR

This paper introduces LanFL, a LAN-aware hierarchical federated learning framework that accelerates training and reduces WAN traffic by performing intra-LAN aggregation with P2P topologies and infrequent inter-LAN cloud aggregation, while preserving accuracy.

ABSTRACT

Federated learning (FL) was designed to enable mobile phones to collaboratively learn a global model without uploading their private data to a cloud server. However, exiting FL protocols has a critical communication bottleneck in a federated network coupled with privacy concerns, usually powered by a wide-area network (WAN). Such a WAN-driven FL design leads to significantly high cost and much slower model convergence. In this work, we propose an efficient FL protocol, which involves a hierarchical aggregation mechanism in the local-area network (LAN) due to its abundant bandwidth and almost negligible monetary cost than WAN. Our proposed FL can accelerate the learning process and reduce the monetary cost with frequent local aggregation in the same LAN and infrequent global aggregation on a cloud across WAN. We further design a concrete FL platform, namely LanFL, that incorporates several key techniques to handle those challenges introduced by LAN: cloud-device aggregation architecture, intra-LAN peer-to-peer (p2p) topology generation, inter-LAN bandwidth capacity heterogeneity. We evaluate LanFL on 2 typical Non-IID datasets, which reveals that LanFL can significantly accelerate FL training (1.5x-6.0x), save WAN traffic (18.3x-75.6x), and reduce monetary cost (3.8x-27.2x) while preserving the model accuracy.

Motivation & Objective

• Motivate reducing WAN bottlenecks and costs in federated learning by leveraging abundant LAN bandwidth.
• Propose a hierarchical LAN-WAN FL design to improve convergence speed and reduce WAN traffic.
• Develop LanFL with LAN-domain aggregation, P2P intra-LAN topologies, and dynamic device selection to handle LAN heterogeneity.
• Evaluate LanFL on Non-IID datasets to quantify speedups, WAN traffic reductions, and cost savings.

Proposed method

• Propose LAN-aware FL with hierarchical aggregation across LAN domains and a central cloud orchestrator.
• Design LanFL with two aggregation topologies for intra-LAN communication: Parameter Server (PS) and Ring-AllReduce, chosen per LAN based on AP topology and throughput.
• Introduce a device-round (RL) and local epochs (E) tuning strategy to balance speed and accuracy in LANs.
• Implement a two-tier FedAvg-like workflow where LAN-domain aggregations occur frequently (intra-LAN) and cloud-level aggregation occurs infrequently (inter-LAN).
• Dynamically select participating devices per LAN to balance throughput across heterogeneous LAN domains.
• Model communication time in LAN via com_T_L using PS or Ring throughput equations, and compute WAN com_T_W via model size and BW, following established FL formulations.
• Experimentally evaluate on FEMNIST and CelebA datasets using LEAF-based Non-IID splits, comparing LanFL against WAN-driven FL baselines.

Experimental results

Research questions

• RQ1 Can LAN-aware hierarchical aggregation accelerate federated learning convergence compared to WAN-only FL?
• RQ2 How does intra-LAN topology (PS vs Ring) impact LAN throughput and overall training time?
• RQ3 How does cross-LAN heterogeneity affect convergence, and can dynamic device selection mitigate stragglers?
• RQ4 What are the WAN traffic and monetary cost implications of LanFL versus traditional FL?
• RQ5 Does LanFL preserve model accuracy under Non-IID data distributions across LANs?

Key findings

• LanFL significantly accelerates model convergence (1.5×–6.0×) and reduces WAN traffic (18.3×–75.6×) and monetary cost (3.8×–27.2×) while preserving accuracy.
• LanFL achieves faster convergence with lower WAN usage across FEMNIST and CelebA compared to WAN-FL baselines.
• Intra-LAN topologies (PS vs Ring) and AP capacity profiling enable higher throughput and better utilization of LAN bandwidth.
• Dynamic LAN-domain device selection mitigates straggler effects from heterogeneity and maintains synchronized progress across LANs.
• The framework relies on locally frequent aggregation within LANs and infrequent global aggregation across WAN, reducing rounds required for convergence.