[Paper Review] CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning

Reent Schlegel, Siddhartha Kumar+2 more|arXiv (Cornell University)|Dec 16, 2021

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes CodedPaddedFL and CodedSecAgg, two novel federated learning schemes that use coding theory to introduce redundancy across devices, enabling straggler mitigation without client drift. CodedPaddedFL achieves a 18× speed-up over conventional FL on MNIST at 95% accuracy, while CodedSecAgg outperforms LightSecAgg by up to 18.7× in speed-up with enhanced security against model inversion attacks.

ABSTRACT

We present two novel federated learning (FL) schemes that mitigate the effect of straggling devices by introducing redundancy on the devices' data across the network. Compared to other schemes in the literature, which deal with stragglers or device dropouts by ignoring their contribution, the proposed schemes do not suffer from the client drift problem. The first scheme, CodedPaddedFL, mitigates the effect of stragglers while retaining the privacy level of conventional FL. It combines one-time padding for user data privacy with gradient codes to yield straggler resiliency. The second scheme, CodedSecAgg, provides straggler resiliency and robustness against model inversion attacks and is based on Shamir's secret sharing. We apply CodedPaddedFL and CodedSecAgg to a classification problem. For a scenario with 120 devices, CodedPaddedFL achieves a speed-up factor of 18 for an accuracy of 95% on the MNIST dataset compared to conventional FL. Furthermore, it yields similar performance in terms of latency compared to a recently proposed scheme by Prakash et al. without the shortcoming of additional leakage of private data. CodedSecAgg outperforms the state-of-the-art secure aggregation scheme LightSecAgg by a speed-up factor of 6.6-18.7 for the MNIST dataset for an accuracy of 95%.

Motivation & Objective

• Address the client drift problem in federated learning caused by ignoring straggling or dropped devices.
• Develop straggler-resilient FL schemes that maintain convergence to the global optimum despite non-iid data and device delays.
• Integrate secure aggregation with straggler mitigation to protect against model inversion attacks.
• Reduce communication and latency overhead while preserving data privacy and model accuracy.
• Ensure scalability and robustness in large-scale, heterogeneous device networks.

Proposed method

• CodedPaddedFL combines one-time padding for data privacy with gradient codes to enable recovery of straggler updates using redundant data across devices.
• CodedSecAgg uses Shamir’s secret sharing to encode local model updates, ensuring that only the sum of gradients is revealed to the server, preventing model inversion.
• Both schemes introduce redundancy via data encoding on devices, allowing the central server to reconstruct global model updates from a subset of devices, even if some are slow or fail.
• Grouping of devices is used to reduce initial communication load, trading off slightly with straggler mitigation efficiency but significantly reducing overall latency.
• The schemes are designed to be compatible with nonlinear problems via kernel embedding, extending applicability beyond linear models.
• Secure aggregation is integrated via additive masking and secret sharing of random seeds, enabling recovery of missing updates without revealing individual contributions.

Experimental results

Research questions

• RQ1 Can coding-based redundancy in device data mitigate straggler effects in federated learning without inducing client drift?
• RQ2 How does combining one-time padding with gradient codes affect privacy and convergence in non-iid data settings?
• RQ3 To what extent can secret sharing and secure aggregation be integrated with straggler-resilient coding to prevent model inversion attacks?
• RQ4 How does grouping of devices impact the trade-off between communication load and straggler mitigation performance?
• RQ5 What is the scalability of the proposed schemes in large networks with high straggler rates?

Key findings

• CodedPaddedFL achieves a 18× speed-up over conventional federated learning for reaching 95% accuracy on the MNIST dataset with 120 devices, without additional data leakage.
• CodedSecAgg outperforms the state-of-the-art LightSecAgg scheme by a speed-up factor of 6.6 to 18.7 on MNIST for 95% accuracy, depending on the number of colluding agents.
• For a network of 1000 devices, CodedSecAgg achieves a 38.9× speed-up over LightSecAgg when only one malicious agent is present, demonstrating strong scalability.
• The additional latency cost of secure aggregation in CodedSecAgg is reduced to 15% compared to CodedPaddedFL at 1000 devices, showing diminishing overhead with scale.
• CodedSecAgg maintains robustness even under high colluding agent counts, achieving a 6.6× speed-up when 60 out of 120 devices are colluding.
• Both schemes converge to the global optimum and avoid client drift, unlike schemes that discard straggler updates, making them more reliable for non-iid data.