[Paper Review] XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning

MyungJae Shin, Chihoon Hwang+4 more|arXiv (Cornell University)|Jun 9, 2020

Privacy-Preserving Technologies in DataComputer Science19 references57 citations

TL;DR

XorMixFL uses XOR-based mixup to privately augment data across devices in a one-shot federated learning setup, improving non-IID performance by generating synthetic samples while preserving privacy.

ABSTRACT

User-generated data distributions are often imbalanced across devices and labels, hampering the performance of federated learning (FL). To remedy to this non-independent and identically distributed (non-IID) data problem, in this work we develop a privacy-preserving XOR based mixup data augmentation technique, coined XorMixup, and thereby propose a novel one-shot FL framework, termed XorMixFL. The core idea is to collect other devices' encoded data samples that are decoded only using each device's own data samples. The decoding provides synthetic-but-realistic samples until inducing an IID dataset, used for model training. Both encoding and decoding procedures follow the bit-wise XOR operations that intentionally distort raw samples, thereby preserving data privacy. Simulation results corroborate that XorMixFL achieves up to 17.6% higher accuracy than Vanilla FL under a non-IID MNIST dataset.

Motivation & Objective

• Address non-IID data distributions in federated learning by creating privacy-preserving synthetic samples.
• Propose a one-shot FL framework (XorMixFL) that uses XOR-based mixup to augment data without exposing raw samples.
• Maintain data privacy while improving global model accuracy on imbalanced label distributions.

Proposed method

• Introduce XorMixup, which encodes mixed samples with bit-wise XOR to protect raw data while creating synthetic samples.
• Use server-side decoding with its own base samples to produce realistic augmented data.
• Apply a one-shot FL framework where devices upload encoded samples, the server decodes to balance label distribution, and a global model is trained from reconstructed data.
• Incorporate p-fold sample blending and averaging of up to p samples per label to extract common features and control privacy/noise.
• Compare XorMixFL against Vanilla FL and MixFL on non-IID MNIST to evaluate accuracy and privacy trade-offs.

Experimental results

Research questions

• RQ1 Can XOR-based mixup (XorMixup) provide privacy-preserving data augmentation in a one-shot FL setting?
• RQ2 Does XorMixFL improve accuracy over Vanilla FL and MixFL under non-IID data distributions while maintaining higher privacy (as measured by sample dissimilarity)?

Key findings

• XorMixFL achieves up to 17.6% higher accuracy than Vanilla FL on non-IID MNIST in one-shot settings.
• XorMixFL can outperform standalone ML by up to around 8.13% in the same non-IID scenario.
• The privacy measure (MDS) indicates XorMixFL provides higher sample privacy than MixFL across tested configurations.
• An ablation shows XorMixFL with XOR-based encoding/decoding maintains competitive accuracy while offering stronger privacy than MixFL.
• Increasing blending depth p and the number of dummy labels M generally raises privacy (MDS) and can affect accuracy in predictable ways.
• XorMixup can achieve comparable accuracy to MixFL while improving privacy, demonstrating the value of XOR-based augmentation for privacy-preserving one-shot FL.