[Paper Review] Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference

Edward Chou, Josh Beal+4 more|arXiv (Cornell University)|Nov 25, 2018

Privacy-Preserving Technologies in Data41 references137 citations

TL;DR

Faster CryptoNets introduces pruning, quantization, and sparse activation-approximation to accelerate encrypted neural network inference using homomorphic encryption, achieving significantly faster runtimes with competitive accuracy on MNIST.

ABSTRACT

Homomorphic encryption enables arbitrary computation over data while it remains encrypted. This privacy-preserving feature is attractive for machine learning, but requires significant computational time due to the large overhead of the encryption scheme. We present Faster CryptoNets, a method for efficient encrypted inference using neural networks. We develop a pruning and quantization approach that leverages sparse representations in the underlying cryptosystem to accelerate inference. We derive an optimal approximation for popular activation functions that achieves maximally-sparse encodings and minimizes approximation error. We also show how privacy-safe training techniques can be used to reduce the overhead of encrypted inference for real-world datasets by leveraging transfer learning and differential privacy. Our experiments show that our method maintains competitive accuracy and achieves a significant speedup over previous methods. This work increases the viability of deep learning systems that use homomorphic encryption to protect user privacy.

Motivation & Objective

• Motivate privacy-preserving ML via homomorphic encryption for encrypted inference in MLaaS scenarios.
• Develop methods to reduce the computational overhead of encrypted neural networks through sparsity and quantized polynomial representations.
• Derive an optimal sparse polynomial activation approximation compatible with HE to minimize error.
• Show practical viability with transfer learning and differential privacy in real-world datasets.

Proposed method

• Introduce network pruning and progressive quantization to enforce sparsity in model weights.
• Represent remaining weights as powers of two to enable sparse polynomial encodings in HE.
• Develop an optimal quantized polynomial approximation of activation functions under power-of-two coefficient constraints.
• Use sparse plaintext-ciphertext multiplication to reduce homomorphic operation counts (HOPs) via monomial multipliers.
• Adopt a feature-extraction framework and differential privacy to improve real-world applicability while maintaining accuracy.
• Evaluate using MNIST and compare HOPs, runtimes, and accuracy against prior encrypted inference methods.

Experimental results

Research questions

• RQ1 Can sparsity and quantized polynomial representations substantially reduce the HOPs and wall-clock time of encrypted neural network inference?
• RQ2 What is the trade-off between activation function expressivity and approximation error under power-of-two coefficient constraints?
• RQ3 How do pruning, quantization, and activation approximation affect accuracy on real-world datasets under HE constraints?
• RQ4 Is the approach viable for single-image encrypted inference and scalable to practical tasks?

Key findings

• Faster CryptoNets achieves large reductions in homomorphic operations across layers, leading to faster encrypted inference than CryptoNets and CryptoDL variants.
• The proposed activation approximations with coefficients as powers of two maintain competitive test accuracy on MNIST.
• Sparse plaintext-ciphertext multiplication enables O(n) complexity for certain monomial multipliers, reducing per-layer HOPs.
• Quantization and pruning reduce multiplications, lowering overall HOPs while preserving inference accuracy.
• Comparison shows Faster CryptoNets with Swish-AQ achieves substantially lower total HOPs and faster inference times than prior encrypted methods, with competitive accuracy.
• The work demonstrates viability of real-world encrypted inference with differential privacy and feature extraction to handle larger datasets.