[Paper Review] Private Machine Learning in TensorFlow using Secure Computation

Morten Dahl, Jason Mancuso+6 more|arXiv (Cornell University)|Oct 18, 2018

Cryptography and Data Security4 references49 citations

TL;DR

The paper presents tf-encrypted, an open-source framework that integrates secure multi-party computation (SPDZ-based) into TensorFlow, enabling private ML via TensorFlow graphs with benchmarking on MNIST.

ABSTRACT

We present a framework for experimenting with secure multi-party computation directly in TensorFlow. By doing so we benefit from several properties valuable to both researchers and practitioners, including tight integration with ordinary machine learning processes, existing optimizations for distributed computation in TensorFlow, high-level abstractions for expressing complex algorithms and protocols, and an expanded set of familiar tooling. We give an open source implementation of a state-of-the-art protocol and report on concrete benchmarks using typical models from private machine learning.

Motivation & Objective

• Introduce tf-encrypted as an accessible platform for private machine learning within TensorFlow.
• Demonstrate integration of secure computation protocols with TensorFlow graphs to enable private inference and training workflows.
• Provide benchmarks showing performance and accuracy trade-offs for private ML models on common tasks.

Proposed method

• Embed secure multi-party computation (SPDZ-style) into TensorFlow as masked and private tensors.
• Use fixed-point encoding for floating-point numbers and support int64 and CRT-based int100 tensors for precision control.
• Implement secure multiplication and other ops via masking and three-server protocol with data pinned to specific devices.
• Leverage TensorFlow’s distributed execution and graph optimizations to reduce networking and improve performance.

Experimental results

Research questions

• RQ1 Can secure computation protocols be effectively integrated into TensorFlow without sacrificing model usability or performance?
• RQ2 What are the performance and accuracy trade-offs when executing private inferences on standard ML models within a TensorFlow-based framework?
• RQ3 How do precision choices (int64 vs int100) impact accuracy and runtime in private ML workloads?
• RQ4 Can a common platform facilitate experimentation with various secure computation optimizations and protocols for private ML?

Key findings

• tf-encrypted enables private inference in TensorFlow with models and inputs kept on separate parties.
• The SPDZ-based protocol, adapted for tensor operations and three-server setup, achieves measurable runtimes suitable for practical use on MNIST-sized networks.
• int64 and int100 fixed-point representations yield similar accuracy, with int100 offering higher precision at additional runtime cost.
• Networks with varying architectures show sub-linear scaling with batch size, highlighting a trade-off between latency and throughput.
• Compared to plaintext TensorFlow, private inference incurs runtime overhead but maintains close accuracy and acceptable KL divergence.
• The framework demonstrates the feasibility of combining private ML with standard TensorFlow tooling, including TensorBoard for debugging and profiling.