[Paper Review] Federated Learning with Server Learning: Enhancing Performance for Non-IID Data

Van Sy Mai, Richard J. La+1 more|arXiv (Cornell University)|Oct 6, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes Federated Server Learning (FSL), a novel approach that enhances federated learning (FL) performance under non-IID data by enabling the central server to perform auxiliary learning on a small, private dataset. Even with limited or distributionally different server data, FSL significantly improves model accuracy and convergence speed compared to conventional FL methods like FedAvg, SCAFFOLD, and FedDyn.

ABSTRACT

Federated Learning (FL) has emerged as a means of distributed learning using local data stored at clients with a coordinating server. Recent studies showed that FL can suffer from poor performance and slower convergence when training data at clients are not independent and identically distributed. Here we consider a new complementary approach to mitigating this performance degradation by allowing the server to perform auxiliary learning from a small dataset. Our analysis and experiments show that this new approach can achieve significant improvements in both model accuracy and convergence time even when the server dataset is small and its distribution differs from that of the aggregated data from all clients.

Motivation & Objective

• To address the performance degradation in federated learning when clients' data are non-IID, a common real-world challenge.
• To explore whether the central server—often assumed to be passive—can actively improve FL performance through auxiliary learning.
• To evaluate the impact of server data size and distributional divergence on FL performance.
• To demonstrate that server learning can be a lightweight, complementary enhancement to existing FL techniques.

Proposed method

• The server trains a local model on a small private dataset, distinct from client data, to serve as a reference for global model updates.
• The global model is updated using a weighted combination of client model updates and server model updates, with a learnable server weight γ to balance contributions.
• The method introduces a regularized optimization objective that incorporates server knowledge via a server loss term, improving alignment with global optima.
• The approach is compatible with existing FL frameworks and can be combined with techniques like SCAFFOLD or FedDyn.
• The server dataset can be real, synthetic, or from simulations, enabling practical deployment even with limited data.
• The method uses a learning rate schedule and regularization to stabilize training and improve convergence.

Experimental results

Research questions

• RQ1 Can a central server improve federated learning performance by performing auxiliary learning on a small dataset?
• RQ2 How does the distributional divergence between server data and client data affect the performance gain?
• RQ3 Does the size of the server dataset influence the improvement in model accuracy and convergence speed?
• RQ4 Can server learning outperform existing state-of-the-art FL algorithms like SCAFFOLD and FedDyn under non-IID conditions?
• RQ5 Is the benefit of server learning robust when the server data are synthetically generated or from a different domain?

Key findings

• FSL achieved 0.6144 test accuracy on CIFAR-10 with C=2 (highly non-IID) after 1,000 rounds, outperforming FedDyn (0.5779) and SCAFFOLD.
• FSL reached 50% accuracy in 203 global rounds on CIFAR-10 with C=2, compared to 502 rounds for FedDyn, demonstrating faster convergence.
• Even with synthetic server data (FSLsyn), the method achieved comparable performance to FSL on EMNIST and slightly worse but still superior results on CIFAR-10.
• Increasing the server data size from 250 to 500 samples improved FSL accuracy from 0.5835 to 0.5845 on CIFAR-10, showing a measurable benefit from more data.
• FSL with real server data (n₀=250) outperformed FSLsyn with larger synthetic data (n₀=720), indicating that data distribution matters more than size alone.
• The method showed robustness to hyperparameter choices, with stable performance across varying learning rates and regularization parameters.