[Paper Review] Model Pruning Enables Localized and Efficient Federated Learning for Yield Forecasting and Data Sharing

Andy Li, Milan Marković+2 more|arXiv (Cornell University)|Apr 19, 2023

Privacy-Preserving Technologies in Data57 references4 citations

TL;DR

This paper proposes Federated Pruning (FedPruning), a communication-efficient federated learning framework that applies neural network pruning to client models before aggregation, enabling localized, privacy-preserving model training for soybean yield forecasting. The method improves local inference performance by 15.5–20% over FedAvg, reduces model size by up to 84%, and cuts communication costs by 57.1–64.7%.

ABSTRACT

Federated Learning (FL) presents a decentralized approach to model training in the agri-food sector and offers the potential for improved machine learning performance, while ensuring the safety and privacy of individual farms or data silos. However, the conventional FL approach has two major limitations. First, the heterogeneous data on individual silos can cause the global model to perform well for some clients but not all, as the update direction on some clients may hinder others after they are aggregated. Second, it is lacking with respect to the efficiency perspective concerning communication costs during FL and large model sizes. This paper proposes a new technical solution that utilizes network pruning on client models and aggregates the pruned models. This method enables local models to be tailored to their respective data distribution and mitigate the data heterogeneity present in agri-food data. Moreover, it allows for more compact models that consume less data during transmission. We experiment with a soybean yield forecasting dataset and find that this approach can improve inference performance by 15.5% to 20% compared to FedAvg, while reducing local model sizes by up to 84% and the data volume communicated between the clients and the server by 57.1% to 64.7%.

Motivation & Objective

• Address the challenge of data heterogeneity in federated learning for agri-food applications, where client data distributions vary significantly across farms.
• Reduce communication overhead and model size in federated learning to enable deployment on resource-constrained edge devices in rural agricultural settings.
• Improve local inference performance by allowing client models to adapt to their unique data distributions through localized pruning.
• Demonstrate that pruning during federated training can simultaneously enhance model efficiency and performance compared to standard FedAvg.
• Enable secure, decentralized collaboration among farms without sharing raw data, supporting sustainable and transparent agri-food systems.

Proposed method

• Apply structured magnitude-based pruning to individual client models before model aggregation in each FL round, reducing model size and communication cost.
• Train client models independently on their local, non-IID data, allowing each to learn a unique, data-specific representation through pruning.
• Aggregate only the pruned client models at the central server to form a global model, preserving privacy and reducing data transmission volume.
• Use a pruning schedule that maintains model performance while maximizing sparsity, with pruning applied per client based on local data characteristics.
• Integrate pruning into the FedAvg framework, creating a new algorithm called FedPruning that retains the communication efficiency of FedAvg while improving local performance.
• Apply pruning policies (e.g., magnitude-based) to remove low-importance weights, resulting in compact, energy-efficient models suitable for edge deployment.

Experimental results

Research questions

• RQ1 Can model pruning in federated learning improve local inference performance on non-IID agricultural data compared to standard FedAvg?
• RQ2 To what extent can pruning reduce model size and communication costs in federated yield forecasting without sacrificing performance?
• RQ3 Does localized pruning enable better generalization on client-specific data distributions in agri-food applications?
• RQ4 How does the performance of pruned federated models compare to centralized training baselines in real-world yield forecasting tasks?
• RQ5 Can FedPruning be effectively deployed on edge devices in rural, low-bandwidth agricultural environments?

Key findings

• FedPruning improved local inference performance by 15.5% to 20% compared to FedAvg across multiple years and pruning policies on a soybean yield forecasting dataset.
• The method reduced local model sizes by up to 84% compared to unpruned models, significantly enhancing memory and energy efficiency.
• Communication costs were reduced by 57.1% to 64.7% due to smaller model parameters transmitted between clients and server in each FL round.
• In most settings, FedPruning achieved performance comparable to or slightly better than the centralized training baseline, despite decentralized training and pruning.
• The results support the hypothesis that moderately pruned models can outperform unpruned models in distributed settings, even with non-IID data.
• The approach enables practical deployment on edge devices in rural agri-food settings due to reduced computational and communication demands.