[Paper Review] LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulation

Joshua C. Zhao, Atul Sharma+4 more|arXiv (Cornell University)|Mar 21, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

LOKI is a novel data reconstruction attack against federated learning that enables a malicious server to recover 76–86% of client data in FedAVG with secure aggregation by injecting customized convolutional kernels. The attack exploits gradient separation through identity-mapping layers, bypassing secure aggregation and enabling large-scale, high-fidelity reconstruction without model inconsistency or prior knowledge of attack timing.

ABSTRACT

Federated learning was introduced to enable machine learning over large decentralized datasets while promising privacy by eliminating the need for data sharing. Despite this, prior work has shown that shared gradients often contain private information and attackers can gain knowledge either through malicious modification of the architecture and parameters or by using optimization to approximate user data from the shared gradients. However, prior data reconstruction attacks have been limited in setting and scale, as most works target FedSGD and limit the attack to single-client gradients. Many of these attacks fail in the more practical setting of FedAVG or if updates are aggregated together using secure aggregation. Data reconstruction becomes significantly more difficult, resulting in limited attack scale and/or decreased reconstruction quality. When both FedAVG and secure aggregation are used, there is no current method that is able to attack multiple clients concurrently in a federated learning setting. In this work we introduce LOKI, an attack that overcomes previous limitations and also breaks the anonymity of aggregation as the leaked data is identifiable and directly tied back to the clients they come from. Our design sends clients customized convolutional parameters, and the weight gradients of data points between clients remain separate even through aggregation. With FedAVG and aggregation across 100 clients, prior work can leak less than 1% of images on MNIST, CIFAR-100, and Tiny ImageNet. Using only a single training round, LOKI is able to leak 76-86% of all data samples.

Motivation & Objective

• To address the lack of scalable, high-fidelity data reconstruction attacks in federated learning under realistic conditions, including FedAVG and secure aggregation.
• To overcome the limitations of prior attacks that fail to scale beyond single clients or small batch sizes.
• To enable the malicious server to reconstruct private data from aggregated gradients while maintaining stealth and avoiding detection via model inconsistency.
• To demonstrate that secure aggregation does not fully protect against data leakage when model parameters are manipulated by the server.
• To show that even with differential privacy, the attack remains effective if applied uniformly across all training steps, due to persistent gradient leakage.

Proposed method

• The server injects a customized convolutional layer with identity-mapping properties into the global model before sending it to clients, ensuring that the weight gradients from each client remain separable after aggregation.
• Each client’s model is modified with a unique convolutional kernel that maps input features to a shared latent space, preserving client-specific gradient signals across aggregation.
• The attack leverages the final convolutional layer and subsequent fully connected layers as a leakage module, enabling precise reconstruction of input data from gradients.
• The method uses a convolutional scaling factor to maintain gradient separability and avoid model inconsistency, even when clients train locally over multiple rounds.
• The server reconstructs data by solving a linear system derived from the shared gradients, using the known structure of the injected convolutional layer.
• The attack is designed to be stealthy, as the injected layers can be hidden within standard architectures like ResNet or VGG, making detection difficult for clients with limited computational verification capabilities.

Experimental results

Research questions

• RQ1 Can a malicious server reconstruct private client data at scale in FedAVG with secure aggregation, where prior attacks fail due to gradient mixing?
• RQ2 How can a server maintain high reconstruction quality while avoiding model inconsistency and detection during federated training?
• RQ3 What architectural modifications enable gradient separation across multiple clients despite secure aggregation?
• RQ4 To what extent does the attack scale with increasing client count and local dataset size?
• RQ5 Can the attack remain effective even when differential privacy is applied uniformly across training steps?

Key findings

• LOKI achieves a data leakage rate of 76–86% across MNIST, CIFAR-100, and Tiny ImageNet using only a single training round with 100 clients under FedAVG and secure aggregation.
• The attack outperforms the state-of-the-art, which leaks less than 1% of data under the same conditions, demonstrating a 76x–86x improvement in reconstruction scale.
• The method maintains high reconstruction quality even after max-pooling, with content clarity preserved despite resolution loss.
• The attack scales linearly with the number of clients and local dataset size, as the required model parameters grow linearly while non-zero parameters remain constant.
• LOKI remains effective even when differential privacy is applied, though it reduces model accuracy due to noise injection, highlighting a trade-off between privacy and utility.
• The attack does not require model inconsistency or repeated iterations, and can be executed in a single round with no need for iterative optimization or blind source separation.