[Paper Review] Secure multiparty computations in floating-point arithmetic

Chuan Guo, Awni Hannun+4 more|arXiv (Cornell University)|Jan 9, 2020

Stochastic Gradient Optimization Techniques10 references11 citations

TL;DR

This paper proposes a secure multiparty computation framework for privacy-preserving machine learning using standard IEEE double-precision floating-point arithmetic. It achieves perfect privacy at a precision of ~10⁻⁵ by rigorously bounding information leakage—less than a millionth of a bit per input—while ensuring numerical stability and high performance on commodity CPUs and GPUs.

ABSTRACT

Secure multiparty computations enable the distribution of so-called shares of sensitive data to multiple parties such that the multiple parties can effectively process the data while being unable to glean much information about the data (at least not without collusion among all parties to put back together all the shares). Thus, the parties may conspire to send all their processed results to a trusted third party (perhaps the data provider) at the conclusion of the computations, with only the trusted third party being able to view the final results. Secure multiparty computations for privacy-preserving machine-learning turn out to be possible using solely standard floating-point arithmetic, at least with a carefully controlled leakage of information less than the loss of accuracy due to roundoff, all backed by rigorous mathematical proofs of worst-case bounds on information loss and numerical stability in finite-precision arithmetic. Numerical examples illustrate the high performance attained on commodity off-the-shelf hardware for generalized linear models, including ordinary linear least-squares regression, binary and multinomial logistic regression, probit regression, and Poisson regression.

Motivation & Objective

• Address the need for privacy-preserving machine learning in sensitive data applications where traditional security measures like passwords are insufficient.
• Overcome limitations of integer-based cryptographic methods by enabling efficient, accurate computations using standard floating-point arithmetic.
• Provide rigorous mathematical bounds on information leakage and numerical stability in finite-precision arithmetic for secure multiparty computation.
• Enable practical deployment of secure multiparty computation for generalized linear models on commodity hardware without specialized optimizations.
• Ensure that intermediate computations remain private under the honest-but-curious threat model, even when parties are curious but follow protocols correctly.

Proposed method

• Uses additive secret sharing to distribute data shares among multiple parties, ensuring no single party can reconstruct the original data.
• Employs Beaver's multiplication protocol to enable secure multiplication in the context of secret-shared values.
• Applies polynomial approximations to implement non-linear functions (e.g., logit, softmax, exp) required in generalized linear models with controlled error.
• Bounds information leakage via rigorous mathematical analysis, showing that leakage is less than the inherent roundoff error in double-precision arithmetic.
• Leverages minibatched stochastic gradient descent for model training, using only basic operations (addition, matrix-vector multiplication, and function evaluation).
• Uses the inverse of the link function (e.g., logistic, probit, exponential) to compute gradients efficiently in the secure computation pipeline.

Experimental results

Research questions

• RQ1 Can secure multiparty computation for machine learning be implemented efficiently using standard floating-point arithmetic without sacrificing numerical accuracy?
• RQ2 What is the maximum amount of information that can leak during secure computation in finite-precision arithmetic, and can it be bounded rigorously?
• RQ3 How can non-linear functions required in generalized linear models be securely approximated in a floating-point setting with minimal error?
• RQ4 Can the proposed scheme achieve both high performance and strong privacy guarantees on commodity hardware like CPUs and GPUs?
• RQ5 To what extent does the information leakage compare to the inherent roundoff error in double-precision floating-point arithmetic?

Key findings

• Information leakage is bounded to less than 10⁻⁶ bits per input real number, which is less than the typical roundoff error of ~10⁻⁵ in double-precision arithmetic.
• The scheme achieves perfect privacy at a precision of approximately 10⁻⁵, where leakage is negligible compared to numerical error.
• For a megapixel image, the total information leakage is at most one bit—insufficient to distinguish fine details, only enough to determine brightness.
• The method supports a wide range of generalized linear models, including linear least squares, logistic regression, probit regression, and Poisson regression.
• The implementation achieves high performance on commodity CPUs and GPUs, avoiding the need for specialized hardware or complex optimizations.
• The framework is compatible with standard machine learning pipelines, using minibatched stochastic gradient descent and secure evaluation of non-linear link functions via polynomial approximation.