[Paper Review] Expressive variational quantum circuits provide inherent privacy in federated learning

Niraj Kumar, Jamie Heredge+4 more|arXiv (Cornell University)|Sep 22, 2023

Stochastic Gradient Optimization Techniques4 citations

TL;DR

This paper proposes a quantum federated learning framework using expressive variational quantum circuits (VQCs) with high-frequency Fourier components to inherently resist gradient inversion attacks. By leveraging overparameterized ansätze and expressive data encoding, the method transforms client data into high-degree multivariate Chebyshev polynomials in the gradient space, creating an exponentially hard optimization problem for attackers, thus providing intrinsic privacy without additional noise or masking.

ABSTRACT

Federated learning has emerged as a viable distributed solution to train machine learning models without the actual need to share data with the central aggregator. However, standard neural network-based federated learning models have been shown to be susceptible to data leakage from the gradients shared with the server. In this work, we introduce federated learning with variational quantum circuit model built using expressive encoding maps coupled with overparameterized ansätze. We show that expressive maps lead to inherent privacy against gradient inversion attacks, while overparameterization ensures model trainability. Our privacy framework centers on the complexity of solving the system of high-degree multivariate Chebyshev polynomials generated by the gradients of quantum circuit. We present compelling arguments highlighting the inherent difficulty in solving these equations, both in exact and approximate scenarios. Additionally, we delve into machine learning-based attack strategies and establish a direct connection between overparameterization in the original federated learning model and underparameterization in the attack model. Furthermore, we provide numerical scaling arguments showcasing that underparameterization of the expressive map in the attack model leads to the loss landscape being swamped with exponentially many spurious local minima points, thus making it extremely hard to realize a successful attack. This provides a strong claim, for the first time, that the nature of quantum machine learning models inherently helps prevent data leakage in federated learning.

Motivation & Objective

• Address the critical privacy vulnerability in classical federated learning where gradients can leak sensitive client data via inversion attacks.
• Explore whether quantum machine learning models, particularly variational quantum circuits, can provide intrinsic privacy without relying on heuristic defenses like gradient noise or masking.
• Investigate the role of quantum circuit expressivity—specifically high-frequency Fourier components—in enhancing privacy during federated training.
• Demonstrate that overparameterization in the FL model and underparameterization in the attack model create a fundamental asymmetry that protects data privacy.
• Provide theoretical and empirical evidence that solving the system of high-degree Chebyshev polynomials derived from quantum gradients is exponentially hard, even for machine learning-based attacks.

Proposed method

• Employ variational quantum circuits (VQCs) with expressive encoding maps that project classical data into a high-dimensional Hilbert space, resulting in models with a large number of non-degenerate Fourier frequencies.
• Use overparameterized ansätze to ensure trainability of the FL model, while maintaining high expressivity in the feature map.
• Model the gradient inversion attack as solving a system of high-degree multivariate Chebyshev polynomials derived from the quantum circuit's parameter shift rule.
• Analyze the computational hardness of solving these polynomial systems both exactly and approximately, showing exponential scaling in the number of qubits.
• Conduct numerical simulations to compare loss landscapes of attack models underparameterized in input variables versus FL models overparameterized in parameters, demonstrating spurious local minima proliferation in the attack model.
• Use univariate quantum models with 2 and 4 qubits to simulate data recovery attacks, showing that higher-frequency models preserve high-frequency components and thus maintain privacy.

Experimental results

Research questions

• RQ1 Can expressive variational quantum circuits in federated learning inherently prevent data leakage from shared gradients without additional privacy mechanisms?
• RQ2 What is the computational complexity of recovering client inputs from gradients in quantum federated learning, particularly when the quantum circuit has high Fourier expressivity?
• RQ3 How does the disparity between overparameterization in the FL model and underparameterization in the attack model affect the feasibility of successful gradient inversion attacks?
• RQ4 To what extent do high-frequency components in quantum feature maps contribute to the robustness of privacy in federated learning?
• RQ5 Can the structure of quantum circuits—specifically their polynomial gradient behavior—be leveraged to create a provably hard inverse problem for attackers?

Key findings

• Expressive quantum circuits with high-frequency Fourier components generate gradients that correspond to high-degree multivariate Chebyshev polynomials, making the inverse problem exponentially hard to solve.
• Theoretical analysis shows that solving the system of equations derived from quantum gradients is intractable both exactly and approximately, due to the exponential growth in the number of terms with qubit count.
• Numerical simulations confirm that when the attack model is underparameterized in input variables, the loss landscape becomes dominated by exponentially many spurious local minima, severely hampering optimization and data recovery.
• In contrast, the FL model remains trainable due to overparameterization in its ansatz, ensuring that high expressivity does not compromise model performance.
• A 4-qubit model preserves high-frequency components of the target function even when fitting a low-frequency signal, whereas a 2-qubit model suppresses such components, leading to reduced privacy.
• The preservation of high-frequency terms in the quantum model’s output ensures that the gradient information remains complex and non-invertible, providing a strong foundation for inherent privacy in federated learning.