[Paper Review] Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning

Dengke Yan, Hu Ming+5 more|arXiv (Cornell University)|Nov 22, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes S²FL, a novel Split Federated Learning framework that enhances both efficiency and accuracy in AIoT systems by introducing an adaptive sliding model split strategy and a data balance-based training mechanism. By dynamically allocating model portions based on device capabilities and recombining features by label to ensure near-IID data distribution, S²FL reduces straggler effects and mitigates client drift, achieving up to 16.5% higher inference accuracy and 3.54× faster training than conventional SFL.

ABSTRACT

Due to its advantages in resource constraint scenarios, Split Federated Learning (SFL) is promising in AIoT systems. However, due to data heterogeneity and stragglers, SFL suffers from the challenges of low inference accuracy and low efficiency. To address these issues, this paper presents a novel SFL approach, named Sliding Split Federated Learning (S$^2$FL), which adopts an adaptive sliding model split strategy and a data balance-based training mechanism. By dynamically dispatching different model portions to AIoT devices according to their computing capability, S$^2$FL can alleviate the low training efficiency caused by stragglers. By combining features uploaded by devices with different data distributions to generate multiple larger batches with a uniform distribution for back-propagation, S$^2$FL can alleviate the performance degradation caused by data heterogeneity. Experimental results demonstrate that, compared to conventional SFL, S$^2$FL can achieve up to 16.5\% inference accuracy improvement and 3.54X training acceleration.

Motivation & Objective

• To address low inference accuracy in Split Federated Learning (SFL) caused by data heterogeneity and non-IID data distributions.
• To improve training efficiency in SFL by mitigating the straggler problem due to device heterogeneity in computing capability and communication speed.
• To design a framework that enables efficient, accurate, and scalable model training on resource-constrained AIoT devices without compromising privacy.
• To integrate adaptive model partitioning and data-aware feature grouping into SFL to enhance convergence and model generalization.

Proposed method

• S²FL splits the global model into three parts: server-side, shared, and client-side model portions to enable flexible and adaptive deployment.
• An adaptive sliding model split strategy assigns model portions to devices based on their computational capacity (FLOPs) and communication bandwidth, balancing training time across devices.
• The data balance-based training mechanism recombines intermediate features from devices by label to form more uniformly distributed batches for training the server-side model portion.
• Multiple copies of the server-side model are trained on these grouped, label-balanced features to reduce performance degradation from non-IID data.
• Gradients from the server-side model are sent back to client devices for local update, and client-side model portions are aggregated via a fed server to form the new global client-side model.
• The framework leverages label information on the main server to enable label-aware feature grouping, improving generalization without exposing raw data.

Experimental results

Research questions

• RQ1 How can model partitioning be dynamically adapted to device-specific capabilities to reduce straggler effects in SFL?
• RQ2 To what extent can data heterogeneity in AIoT devices degrade SFL performance, and how can it be mitigated through feature reorganization?
• RQ3 Can a hybrid training strategy combining adaptive model splitting and data-balanced feature grouping improve both accuracy and efficiency in SFL?
• RQ4 How does S²FL perform under varying device compositions, including diverse FLOPs and communication rates?
• RQ5 What is the relative contribution of the adaptive sliding model split and data balance-based training mechanism to overall performance gains?

Key findings

• S²FL achieves up to 16.5% higher inference accuracy compared to conventional SFL under non-IID data settings.
• S²FL reduces training time by 3.54× and communication overhead by 2.57× compared to SFL when the data skew parameter a=0.5.
• The ablation study confirms that both the adaptive sliding model split and data balance-based training mechanism independently improve performance, with the full S²FL+MB configuration yielding the highest accuracy and fastest convergence.
• With increasing numbers of participating devices (5 to 20), S²FL consistently outperforms baselines, showing robustness and scalability.
• In heterogeneous device compositions—especially with higher proportions of low-FLOP and low-bandwidth devices—S²FL maintains faster convergence and higher accuracy than SFL.
• The data balance-based mechanism improves test accuracy over SFL by reducing the impact of non-IID data, while the adaptive split strategy accelerates training by minimizing straggler delays.