[Paper Review] TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service

Amartya Sanyal, Matt J. Kusner+2 more|arXiv (Cornell University)|Jun 9, 2018

Cryptography and Data SecurityComputer Science30 references40 citations

TL;DR

This paper presents TAPAS, a set of techniques to run binary neural networks on fully homomorphic encryption to enable fast encrypted prediction as a service, with privacy guarantees and various speed-up tricks.

ABSTRACT

Machine learning methods are widely used for a variety of prediction problems. \emph{Prediction as a service} is a paradigm in which service providers with technological expertise and computational resources may perform predictions for clients. However, data privacy severely restricts the applicability of such services, unless measures to keep client data private (even from the service provider) are designed. Equally important is to minimize the amount of computation and communication required between client and server. Fully homomorphic encryption offers a possible way out, whereby clients may encrypt their data, and on which the server may perform arithmetic computations. The main drawback of using fully homomorphic encryption is the amount of time required to evaluate large machine learning models on encrypted data. We combine ideas from the machine learning literature, particularly work on binarization and sparsification of neural networks, together with algorithmic tools to speed-up and parallelize computation using encrypted data.

Motivation & Objective

• Motivate prediction as a service under strict client data privacy and minimal client/server interaction.
• Propose a framework to perform ML inference on encrypted data using binary neural networks (BNNs) with FHE.
• Develop Boolean circuits and architectural tricks to speed up encrypted inner products, activations, and batching operations.
• Offer practical, parallelizable methods to update models without re-encrypting client data and to sparsify encrypted computations.

Proposed method

• Use TFHE-based FHE that supports binary operations to execute BNN layers on encrypted data.
• Design two inner-product circuits: reduce-tree and sorting-network; compare runtimes to choose the more efficient option.
• Implement an encrypted inner product via XNOR followed by a sum (popcount) using shallow circuits (reduce-tree or sorting network).
• Develop a comparator circuit to enforce the bias condition 2S >= d - b for activations with possible batch-normalization bias.
• Introduce the +1 trick to sparsify encrypted computations by exploiting binary weight structure to reduce the number of encrypted additions.
• Apply weight ternarization (−1,0,1) to drop connections and further reduce encrypted computation when advantageous.
• Demonstrate parallelizable evaluation across outputs and datasets, including Cancer, Diabetes, Faces, and MNIST.

Experimental results

Research questions

• RQ1 Can binary neural networks be efficiently evaluated on fully homomorphic encrypted data while maintaining high accuracy?
• RQ2 What circuit designs and tricks (reduce tree vs sorting network, +1 trick, sparsification) yield practical speedups for encrypted predictions?
• RQ3 How do privacy guarantees translate into a practical EPAAS (Encrypted Prediction as a Service) workflow, with minimal client interaction and model updates?
• RQ4 What is the impact of model type (binary/ternary) and network depth on accuracy and latency under FHE constraints?

Key findings

• BNN-based encrypted predictions achieve competitive accuracy on several datasets (Cancer, Diabetes, Faces, MNIST) compared with floating-point networks.
• A reduce-tree inner-product circuit outperforms a sorting-network-based approach in parallelized settings for encrypted computation.
• The +1 trick effectively halves the computation time per layer by sparsifying encrypted sums.
• Ternarizing weights (−1,0,1) can reduce computation with modest accuracy loss, depending on dataset.
• Full parallelization across outputs dramatically reduces latency, with substantial gains shown across datasets.
• On MNIST, the approach achieves competitive accuracy to floating-point networks, and the method enables practical, privacy-preserving predictions.