[论文解读] ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge

Xiaohong Yang, Tong Xie+6 more|arXiv (Cornell University)|Jan 20, 2026

Privacy-Preserving Technologies in Data被引用 0

一句话总结

ELSA 提出一种将拆分学习与分层联邦学习相结合的框架，在网络边缘实现隐私感知、资源高效的大模型微调，并引入聚类、自适应模型分割和轻量级通信以应对异质性、隐私和效率约束。

ABSTRACT

Training large language models (LLMs) at the network edge faces fundamental challenges arising from device resource constraints, severe data heterogeneity, and heightened privacy risks. To address these challenges, we propose ELSA (Efficient LLM-centric Split Aggregation), a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric Kullback-Leibler (KL) divergence, augmented by prediction-consistency trust scoring and latency-aware edge assignment to jointly mitigate data heterogeneity, device unreliability, and communication constraints. Second, it employs a resource-aware dynamic model splitting strategy to adaptively partition the LLM into three segments across clients and edge servers, with the cloud used only for adapter aggregation, enabling an effective balance between on-device computation cost and global convergence stability. Third, it incorporates a lightweight communication scheme based on computational sketches combined with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead while mitigating privacy leakage during model exchanges across the network. Extensive experiments across diverse NLP tasks demonstrate that ELSA consistently outperforms state-of-the-art baselines in terms of adaptability, convergence behavior, and robustness, establishing a scalable and privacy-aware solution for edge-side LLM fine-tuning under resource constraints.

研究动机与目标

• 解决边缘端大模型微调中的资源约束、数据异质性与隐私风险。
• 提出一个将拆分学习与分层联邦学习整合的统一框架，以应用于边缘网络。
• 提供客户端聚类、自适应模型分区、低开销通信等机制，以提升收敛性与鲁棒性。
• 展示在多样化NLP任务中的可扩展性与隐私保护能力。

提出的方法

• 引入一个与任务无关、对行为敏感的客户端聚类，使用公开探测输入的语义指纹和对称KL散度。
• 结合预测一致性信任评分与延迟感知的边缘分配，以缓解异质性、不可靠性和通信约束。
• 应用资源感知的动态模型分割策略，将大模型在客户端和边缘服务器之间分成三段，云端仅用于适配器聚合。
• 开发一种轻量级通信方案，使用计算草图和语义子空间正交扰动（SS-OP），以降低开销并在交换过程中减轻隐私泄漏。

实验结果

研究问题

• RQ1 在资源受限的边缘设备上，如何在保/privacy 的前提下有效微调大模型？
• RQ2 分层联邦 + 拆分学习的方法能否在数据异质性和设备不可靠性下提升收敛性与鲁棒性？
• RQ3 哪些聚类、分区和通信技术能够在边缘端计算、边缘聚合与云端参与之间实现对大模型微调的最佳平衡？
• RQ4 基于 SS-OP 的通信是否在降低带宽的同时保持隐私？

主要发现

• ELSA 在多样化的 NLP 任务上，在适应性、收敛行为和鲁棒性方面持续优于最先进的基线。
• 该框架实现了边端计算成本与全局收敛稳定性之间的有效平衡。
• 所提出的聚类、分割与通信策略缓解了数据异质性、设备不可靠性和通信约束。