[Paper Review] Secret Sharing based Secure Regressions with Applications

Chaochao Chen, Liang Li+7 more|arXiv (Cornell University)|Apr 10, 2020

Cryptography and Data Security27 references4 citations

TL;DR

This paper proposes secure multiparty regression protocols based on secret sharing for linear and logistic regression, enabling privacy-preserving collaboration among data holders. By reducing model training to secure matrix summation and two-party multiplication, the method achieves identical accuracy to plaintext models with linearly scaling communication and computation costs, demonstrating scalability on large datasets and real-world deployment in financial risk control.

ABSTRACT

Nowadays, the utilization of the ever expanding amount of data has made a huge impact on web technologies while also causing various types of security concerns. On one hand, potential gains are highly anticipated if different organizations could somehow collaboratively share their data for technological improvements. On the other hand, data security concerns may arise for both data holders and data providers due to commercial or sociological concerns. To make a balance between technical improvements and security limitations, we implement secure and scalable protocols for multiple data holders to train linear regression and logistic regression models. We build our protocols based on the secret sharing scheme, which is scalable and efficient in applications. Moreover, our proposed paradigm can be generalized to any secure multiparty training scenarios where only matrix summation and matrix multiplications are used. We demonstrate our approach by experiments which shows the scalability and efficiency of our proposed protocols, and finally present its real-world applications.

Motivation & Objective

• Address the challenge of enabling collaborative machine learning across multiple data holders while preserving data privacy.
• Develop efficient and scalable protocols for training linear and logistic regression models without exposing raw data.
• Overcome limitations of prior secure multiparty computation methods that do not generalize well to multiple parties.
• Enable practical deployment of secure regression in real-world applications such as risk control and intelligent lending.
• Demonstrate that secure training can achieve the same model performance as plaintext training with only linearly increasing overhead.

Proposed method

• Reduce multiparty regression training to secure multiparty matrix summation and secure two-party matrix multiplication using secret sharing.
• Use secret sharing to protect data inputs, model parameters, and labels during computation, ensuring no party learns sensitive information.
• Treat secure two-party matrix multiplication as a black-box protocol, enabling generalization to any training process relying on summation and multiplication.
• Apply polynomial approximation (3rd order) to handle non-linear functions like sigmoid, transforming them into operations compatible with secret sharing.
• Implement two variants: with and without a trusted initializer, to support different deployment scenarios.
• Use mini-batch gradient descent with adaptive learning rate search for model optimization under privacy constraints.

Experimental results

Research questions

• RQ1 Can secure multiparty regression be achieved with minimal computational and communication overhead while preserving data privacy?
• RQ2 How does the performance of secret sharing-based regression compare to plaintext regression in terms of accuracy and efficiency?
• RQ3 Can the proposed protocol scale linearly with increasing data size in both horizontally and vertically partitioned data settings?
• RQ4 What is the impact of using a trusted initializer on communication and computation costs in secure regression?
• RQ5 Can the protocol be generalized beyond linear and logistic regression to other machine learning models relying on matrix operations?

Key findings

• The proposed secure linear and logistic regression protocols achieve identical performance (RMSE and AUC) to their plaintext counterparts on all benchmark datasets.
• Running times for secure models scale linearly with data size, confirming the method's scalability for large-scale datasets.
• On the News dataset, Sec-LiRe-OTI-H took 78.52 seconds compared to 32.41 seconds for plaintext linear regression, a 2.4-fold increase, which is acceptable given privacy guarantees.
• For logistic regression on the APS dataset, Sec-LoRe-OTI-H took 1,374.45 seconds—9.06 times longer than plaintext LoRe—due to additional rounds for polynomial approximation.
• The protocol with a trusted initializer (TI) reduced communication cost in vertical partitioning by avoiding feature sharing, improving efficiency.
• The method has been successfully deployed in real-world applications, including online risk control and automated lending at CDFinance and Ant Financial.