[Paper Review] Secure multi-party linear regression at plaintext speed

Jonathan M. Bloom|arXiv (Cornell University)|Jan 28, 2019

Genetic Associations and Epidemiology4 citations

TL;DR

This paper presents a secure multi-party computation (SMC) framework for linear regression that achieves plaintext-level efficiency by compressing data within each party and combining compressed statistics across parties. The method enables privacy-preserving, scalable genome-wide association studies (GWAS) with incremental updates for new data, achieving asymptotic efficiency independent of sample size while preserving exact statistical results through secure aggregation of QR-decomposed covariates.

ABSTRACT

We detail distributed algorithms for scalable, secure multiparty linear regression and feature selection at essentially the same speed as plaintext regression. While the core geometric ideas are simple, the recognition of their broad utility when combined is novel. Our scheme opens the door to efficient and secure genome-wide association studies across multiple biobanks.

Motivation & Objective

• To enable secure, scalable, and efficient multi-party linear regression across distributed data centers, hospitals, or biobanks without sharing raw data.
• To address the challenge of performing large-scale genome-wide association studies (GWAS) while preserving privacy and minimizing communication and computation costs.
• To develop a method that supports incremental updates when new data batches or institutions join, with cost independent of original data size.
• To extend the approach to efficiently test millions of genetic variants for association with traits while adjusting for fixed covariates, including principal components.
• To enable secure computation of PCA and regression on combined data without relying on reference panels, overcoming prior scalability limitations.

Proposed method

• Each party performs local compression by computing inner products: $ y_p^ op y_p $, $ C_p^ op y_p $, and $ C_p^ op C_p $, reducing data dimension from samples to covariates.
• The parties securely aggregate these compressed statistics across parties to reconstruct global $ C^ op C $, $ C^ op y $, and $ y^ op y $.
• The method uses QR decomposition of each party’s covariate matrix $ C_p $, producing $ R_p $, which is shared securely to enable global $ R $-matrix reconstruction.
• The global $ R $ matrix is used to compute $ Q^ op y $ and $ Q^ op X $ via $ (R^{-1})^ op $, enabling exact coefficient estimation without revealing raw data.
• The approach supports efficient feature scanning by reusing pre-computed $ R_p $ matrices, allowing rapid re-analysis with different response or covariate sets.
• It integrates naturally with linear mixed models via kinship matrix eigendecomposition and supports iterative optimization in generalized linear models and deep learning.

Experimental results

Research questions

• RQ1 Can secure multi-party computation be used to compute exact linear regression statistics with performance approaching that of plaintext computation in large-scale, distributed settings?
• RQ2 How can privacy-preserving GWAS be performed efficiently when data is distributed across multiple institutions with no data sharing?
• RQ3 Can incremental updates be supported in secure regression with cost independent of the original number of samples?
• RQ4 What is the role of QR decomposition and compressed statistics in enabling secure, scalable, and efficient multi-party regression?
• RQ5 How can the method be extended to support multiple responses, transient covariates, interaction terms, and gene burden tests?

Key findings

• The algorithm achieves asymptotic computational efficiency matching plaintext regression, with the combine step independent of sample size and only dependent on the number of covariates.
• Secure aggregation of compressed statistics (via SMC) ensures that parties learn only the final regression statistics, preserving privacy.
• The method supports incremental updates for new data batches at a cost proportional only to the new batch size, not the original data size.
• The approach enables exact GWAS on combined data without data sharing, using only compressed representations and secure computation.
• The framework efficiently supports feature scanning, interaction terms, and gene burden tests through linear projection and matrix associativity.
• The method generalizes to linear mixed models by leveraging eigendecomposition of kinship matrices, reducing to a single-party scan.