[Paper Review] CrypTen: Secure Multi-Party Computation Meets Machine Learning

Brian Knott, Shobha Venkataraman+4 more|arXiv (Cornell University)|Sep 2, 2021

Cryptography and Data Security64 references62 citations

TL;DR

CrypTen is a PyTorch-like framework that enables private ML by implementing secure MPC with GPU-accelerated, multi-party computations, allowing private training and inference on distributed data. It benchmarks text, speech, and image models under a semi-honest threat model.

ABSTRACT

Secure multi-party computation (MPC) allows parties to perform computations on data while keeping that data private. This capability has great potential for machine-learning applications: it facilitates training of machine-learning models on private data sets owned by different parties, evaluation of one party's private model using another party's private data, etc. Although a range of studies implement machine-learning models via secure MPC, such implementations are not yet mainstream. Adoption of secure MPC is hampered by the absence of flexible software frameworks that "speak the language" of machine-learning researchers and engineers. To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks. This paper describes the design of CrypTen and measure its performance on state-of-the-art models for text classification, speech recognition, and image classification. Our benchmarks show that CrypTen's GPU support and high-performance communication between (an arbitrary number of) parties allows it to perform efficient private evaluation of modern machine-learning models under a semi-honest threat model. For example, two parties using CrypTen can securely predict phonemes in speech recordings using Wav2Letter faster than real-time. We hope that CrypTen will spur adoption of secure MPC in the machine-learning community.

Motivation & Objective

• Promote adoption of secure MPC in machine learning by providing a flexible, ML-first framework.
• Enable private training and inference on private datasets held by multiple parties while preserving data privacy.
• Integrate secure MPC primitives with PyTorch-like tensor operations, autograd, and GPU acceleration.

Proposed method

• Introduce a machine-learning first API that mirrors PyTorch to ease adoption.
• Use eager execution and a CrypTensor abstraction to wrap secret-shared computations with autograd support.
• Implement arithmetic and binary secret sharing with conversions and Beaver triples for secure multiplication.
• Off-load heavy computations to GPUs using CUDA libraries with fixed-point encoding for MPC.
• Provide a semi-honest, multi-party protocol that supports an arbitrary number of parties and practical private inference.

Experimental results

Research questions

• RQ1 How can secure MPC be designed to be as usable as mainstream ML frameworks for researchers and engineers?
• RQ2 What performance can CrypTen achieve for private inference and training on state-of-the-art ML models across text, speech, and image tasks?
• RQ3 How does GPU off-loading and high-performance communication affect private ML workloads under semi-honest security?
• RQ4 What are the practical bottlenecks and trade-offs when scaling to more parties and larger models?
• RQ5 Can CrypTen enable real-time or near real-time private predictions for realistic ML tasks?

Key findings

• CrypTen enables private inference and training with a PyTorch-like API and GPU off-loading, achieving practical speeds on modern models.
• Two-party private inference can classify images in a few seconds for certain models, demonstrating real-time potential under MPC.
• Private inference for speech with Wav2Letter is significantly slower than plaintext PyTorch, but GPU acceleration provides large speedups over CPU.
• ResNet-18 and ViT-B/16 private evaluation on ImageNet are achievable within a few seconds per sample, with communication overhead dominated by activation tensors.
• The framework demonstrates that secure MPC can be practical for modern ML tasks across text, speech, and vision domains under semi-honest security.