[Paper Review] Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning Architectures in a Non-IID Scenario

Pedro Miguel Sánchez Sánchez, Alberto Huertas Celdrán+5 more|arXiv (Cornell University)|Oct 20, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes three decentralized federated learning architectures—HoriChain (horizontal), VertiChain, and VertiComb (vertical)—to evaluate robustness under non-IID data and adversarial attacks. It demonstrates that HoriChain outperforms vertical architectures in resisting data poisoning and gradient poisoning attacks, especially under moderate attack intensities, making it the most robust configuration in decentralized FL settings.

ABSTRACT

Federated learning (FL) allows participants to collaboratively train machine and deep learning models while protecting data privacy. However, the FL paradigm still presents drawbacks affecting its trustworthiness since malicious participants could launch adversarial attacks against the training process. Related work has studied the robustness of horizontal FL scenarios under different attacks. However, there is a lack of work evaluating the robustness of decentralized vertical FL and comparing it with horizontal FL architectures affected by adversarial attacks. Thus, this work proposes three decentralized FL architectures, one for horizontal and two for vertical scenarios, namely HoriChain, VertiChain, and VertiComb. These architectures present different neural networks and training protocols suitable for horizontal and vertical scenarios. Then, a decentralized, privacy-preserving, and federated use case with non-IID data to classify handwritten digits is deployed to evaluate the performance of the three architectures. Finally, a set of experiments computes and compares the robustness of the proposed architectures when they are affected by different data poisoning based on image watermarks and gradient poisoning adversarial attacks. The experiments show that even though particular configurations of both attacks can destroy the classification performance of the architectures, HoriChain is the most robust one.

Motivation & Objective

• To address the lack of robustness evaluation in decentralized vertical federated learning (VFL) architectures under non-IID data and adversarial attacks.
• To compare the resilience of horizontal and vertical FL architectures against data poisoning and gradient poisoning attacks.
• To design and implement three novel decentralized FL architectures—HoriChain, VertiChain, and VertiComb—tailored for horizontal and vertical FL scenarios.
• To evaluate the impact of adversarial attacks on model performance using metrics like accuracy, F1-score, confusion matrices, and client relevance.
• To identify architectural configurations that enhance robustness in federated learning under realistic, privacy-preserving, non-IID data conditions.

Proposed method

• Design and implementation of three decentralized FL architectures: HoriChain for horizontal FL, VertiChain and VertiComb for vertical FL, each with distinct neural network structures and training protocols.
• Deployment of a federated, privacy-preserving handwritten digit classification use case using non-IID data distributions across clients.
• Application of two adversarial attack types: data poisoning via image watermarking and gradient poisoning via manipulated model updates.
• Systematic variation of attack parameters, including gradient multipliers (-1, -10, 0), to assess attack intensity and impact on model convergence.
• Evaluation using multiple metrics: classification accuracy, F1-score, confusion matrices, and client relevance to assess model behavior and attack detectability.
• Comparison of architectural performance under identical attack conditions to determine relative robustness across HFL and VFL settings.

Experimental results

Research questions

• RQ1 How does the robustness of decentralized horizontal federated learning (HFL) compare to vertical federated learning (VFL) under data poisoning attacks using image watermarking?
• RQ2 What is the impact of gradient poisoning attacks with varying intensities (gradient multipliers) on the convergence and performance of HFL and VFL architectures?
• RQ3 How do non-IID data distributions affect the resilience of HFL and VFL architectures under adversarial conditions?
• RQ4 Can the presence of data poisoning or gradient poisoning adversaries be detected using standard metrics like accuracy, F1-score, or confusion matrices?
• RQ5 Which architectural design—HoriChain, VertiChain, or VertiComb—demonstrates the highest resilience to both data and gradient poisoning attacks in decentralized FL settings?

Key findings

• HoriChain demonstrated superior robustness against gradient poisoning attacks with a multiplier of -1, maintaining learnable performance despite delayed convergence, while VertiComb deteriorated to near-random predictions.
• With a gradient multiplier of -10, both HoriChain and VertiComb architectures failed to learn a functional model, indicating complete attack success under high-intensity gradient poisoning.
• Under data poisoning via watermarking, both HoriChain and VertiComb maintained baseline-level accuracy and F1-score on unmarked samples, indicating undetected backdoor behavior and high attack success.
• The confusion matrices revealed that data poisoning attacks were undetectable by standard metrics, as they did not alter performance on clean samples, but significantly impacted model behavior on poisoned inputs.
• HoriChain showed greater resilience than VertiComb under moderate gradient poisoning (multiplier -1), suggesting architectural design plays a critical role in adversarial resistance.
• No significant difference in robustness was observed between HoriChain and VertiComb under zero-gradient multiplier attacks, indicating that minimal poisoning does not disrupt training in either architecture.