[Paper Review] FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions

Shaohua Li, Kaiping Xue+5 more|arXiv (Cornell University)|Nov 20, 2018

Privacy-Preserving Technologies in Data28 references11 citations

TL;DR

FALCON presents a novel, fast, and secure method for evaluating Convolutional Neural Networks (CNNs) using Fourier Transform-based optimization and fully homomorphic encryption. It enables efficient, privacy-preserving inference with the first practical protocol for secure softmax computation, outperforming prior work by over 25× in speed and reducing communication by 97% on MNIST and CIFAR-10 models.

ABSTRACT

Machine learning as a service has been widely deployed to utilize deep neural network models to provide prediction services. However, this raises privacy concerns since clients need to send sensitive information to servers. In this paper, we focus on the scenario where clients want to classify private images with a convolutional neural network model hosted in the server, while both parties keep their data private. We present FALCON, a fast and secure approach for CNN predictions based on Fourier Transform. Our solution enables linear layers of a CNN model to be evaluated simply and efficiently with fully homomorphic encryption. We also introduce the first efficient and privacy-preserving protocol for softmax function, which is an indispensable component in CNNs and has not yet been evaluated in previous works due to its high complexity. We implemented the FALCON and evaluated the performance on real-world CNN models. The experimental results show that FALCON outperforms the best known works in both computation and communication cost.

Motivation & Objective

• To address privacy concerns in machine learning as a service, where clients and servers must protect input data and model parameters.
• To enable secure and efficient inference on CNNs without exposing sensitive data or model weights.
• To design the first efficient, privacy-preserving protocol for the computationally complex softmax layer in CNNs.
• To significantly reduce computation and communication overhead in secure CNN inference compared to existing methods.

Proposed method

• Leverages the linear properties of the Fast Fourier Transform (FFT) to optimize fully homomorphic encryption evaluation in convolutional and fully-connected layers.
• Applies lattice-based fully homomorphic encryption to securely evaluate linear layers with minimal ciphertext multiplication overhead.
• Introduces a novel, efficient protocol for computing the softmax function in a privacy-preserving manner, overcoming prior limitations due to exponentiation and division.
• Uses Yao’s Garbled Circuits for secure evaluation of non-linear layers, including ReLU and max pooling, with optimized processing pipelines.
• Employs fixed-point arithmetic with scaling to represent decimal values as integers, minimizing accuracy loss during encrypted computation.
• Implements a hybrid architecture combining homomorphic encryption and secure two-party computation to balance efficiency and security.

Experimental results

Research questions

• RQ1 Can the Fast Fourier Transform be used to significantly accelerate secure evaluation of linear layers in CNNs under fully homomorphic encryption?
• RQ2 Is it possible to design an efficient, privacy-preserving protocol for the softmax function, which has been overlooked in prior secure deep learning works?
• RQ3 How does the proposed FALCON framework compare in performance and communication cost to existing secure inference systems like MiniONN and GAZELLE?
• RQ4 To what extent does fixed-point approximation affect the prediction accuracy of encrypted CNN models?

Key findings

• FALCON achieves over 25× speedup in online phase computation compared to MiniONN on the CIFAR-10 model, with 97% reduction in communication overhead.
• On the MNIST model, FALCON is over 27× faster than MiniONN in the online phase while reducing communication by more than 97%.
• FALCON outperforms GAZELLE in online computation and communication, despite inconsistencies in GAZELLE’s reported benchmarks that cast doubt on its performance claims.
• The model maintains near-identical accuracy to plaintext inference: 99.26% on MNIST and 81.61% on CIFAR-10, with negligible loss due to fixed-point scaling.
• FALCON is the first framework to provide a secure, efficient, and practical protocol for the softmax layer in encrypted CNN inference.
• The use of FFT and optimized homomorphic encryption reduces the number of expensive ciphertext multiplications, leading to substantial performance gains across all CNN layers.