[Paper Review] Scalable and Privacy-Preserving Federated Principal Component Analysis

David Froelicher, Hyunghoon Cho+7 more|arXiv (Cornell University)|Mar 31, 2023

Biometric Identification and Security4 citations

TL;DR

This paper proposes sf-pca, a scalable and privacy-preserving federated PCA system that uses multiparty homomorphic encryption and edge computing to securely compute principal components on distributed, private data without revealing raw inputs or intermediate results. It achieves centralized-level accuracy, scales linearly with data size and number of parties, and is 3x to 250x faster than prior privacy-preserving alternatives.

ABSTRACT

Principal component analysis (PCA) is an essential algorithm for dimensionality reduction in many data science domains. We address the problem of performing a federated PCA on private data distributed among multiple data providers while ensuring data confidentiality. Our solution, SF-PCA, is an end-to-end secure system that preserves the confidentiality of both the original data and all intermediate results in a passive-adversary model with up to all-but-one colluding parties. SF-PCA jointly leverages multiparty homomorphic encryption, interactive protocols, and edge computing to efficiently interleave computations on local cleartext data with operations on collectively encrypted data. SF-PCA obtains results as accurate as non-secure centralized solutions, independently of the data distribution among the parties. It scales linearly or better with the dataset dimensions and with the number of data providers. SF-PCA is more precise than existing approaches that approximate the solution by combining local analysis results, and between 3x and 250x faster than privacy-preserving alternatives based solely on secure multiparty computation or homomorphic encryption. Our work demonstrates the practical applicability of secure and federated PCA on private distributed datasets.

Motivation & Objective

• Address the lack of secure, accurate, and scalable federated PCA solutions for privacy-sensitive data in domains like genomics and healthcare.
• Overcome the limitations of existing meta-analysis approaches that sacrifice accuracy by combining local PCA results.
• Enable end-to-end privacy in federated PCA where data remains local and intermediate results are encrypted, even from the aggregator.
• Develop a system that scales efficiently with both dataset dimensions and the number of data providers while preserving security under a passive-adversary model.
• Provide a practical, deployable solution for high-complexity linear algebra operations in federated analytics using optimized cryptographic techniques.

Proposed method

• Leverages multiparty homomorphic encryption (MHE) to securely compute on encrypted data across multiple data providers (DPs) without decryption.
• Integrates edge computing to interleave local cleartext computations with collective encrypted operations, reducing communication and latency.
• Employs a hybrid cryptographic approach, switching between MHE and secret sharing for different operations—MHE for large-scale matrix operations, secret sharing for non-polynomial functions on small inputs.
• Uses a randomized PCA (RPCA) algorithm as the core computation, ensuring high accuracy comparable to centralized PCA.
• Optimizes key linear algebra operations such as matrix multiplication, factorization, and orthogonalization using MHE-optimized routines.
• Supports fault tolerance via threshold secret sharing of the MHE secret key, enabling computation to continue if some DPs become unavailable.

Figure 1: Randomized PCA Workflow. Matrix dimensions are shown with the box sizes and are indicated on the left and top of the corresponding box.

Experimental results

Research questions

• RQ1 Can a federated PCA system achieve accuracy comparable to centralized PCA while preserving end-to-end privacy for both raw data and intermediate results?
• RQ2 How can secure multiparty computation be optimized to scale efficiently with the number of data providers and input dimensions in PCA?
• RQ3 Can hybrid cryptographic techniques (MHE and secret sharing) be effectively composed to improve performance on mixed workloads in federated linear algebra?
• RQ4 To what extent does the proposed system outperform existing privacy-preserving PCA solutions in terms of speed, accuracy, and communication overhead?
• RQ5 Is it feasible to deploy a secure, scalable, and accurate federated PCA system in real-world applications involving sensitive data, such as genomics?

Key findings

• sf-pca achieves PCA results that are as accurate as non-secure centralized solutions, regardless of data distribution among data providers.
• The system scales linearly or better with both the number of data providers and the dimensionality of the input data.
• sf-pca is between 3x and 250x faster than existing privacy-preserving alternatives based solely on secure multiparty computation or homomorphic encryption.
• In a two-party setting, sf-pca maintains nearly constant runtime as data size increases, while SMC-based solutions become 1.9 times slower.
• On a genomic dataset of 2,504 individuals with 1,773 features, sf-pca accurately recovers the true low-dimensional structure of the data, matching centralized PCA, while meta-analysis approaches distort the data landscape.
• Each data provider sees only its own portion of the projected output, ensuring confidentiality under the security model, even when sharing results with the aggregator.

Figure 2: sf-pca System Model and Functionality. Each DP i holds a $n_{i}\times m$ submatrix as input and collaboratively executes sf-pca to obtain encrypted PCs and/or the projection of its local data.