[Paper Review] AutoFHE: Automated Adaption of CNNs for Efficient Evaluation over FHE

Wei Ao, Vishnu Naresh Boddeti|arXiv (Cornell University)|Oct 12, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

AutoFHE proposes an automated, multi-objective framework to adapt standard CNNs for efficient homomorphic evaluation under RNS-CKKS by jointly optimizing layerwise mixed-degree polynomial approximations and bootstrapping placement. It achieves up to 1.8× faster inference and 2.56% higher accuracy than low-degree methods, and 103× speedup over TFHE-based approaches, while supporting any CNN architecture with flexible accuracy-latency trade-offs.

ABSTRACT

Secure inference of deep convolutional neural networks (CNNs) under RNS-CKKS involves polynomial approximation of unsupported non-linear activation functions. However, existing approaches have three main limitations: 1) Inflexibility: The polynomial approximation and associated homomorphic evaluation architecture are customized manually for each CNN architecture and do not generalize to other networks. 2) Suboptimal Approximation: Each activation function is approximated instead of the function represented by the CNN. 3) Restricted Design: Either high-degree or low-degree polynomial approximations are used. The former retains high accuracy but slows down inference due to bootstrapping operations, while the latter accelerates ciphertext inference but compromises accuracy. To address these limitations, we present AutoFHE, which automatically adapts standard CNNs for secure inference under RNS-CKKS. The key idea is to adopt layerwise mixed-degree polynomial activation functions, which are optimized jointly with the homomorphic evaluation architecture in terms of the placement of bootstrapping operations. The problem is modeled within a multi-objective optimization framework to maximize accuracy and minimize the number of bootstrapping operations. AutoFHE can be applied flexibly on any CNN architecture, and it provides diverse solutions that span the trade-off between accuracy and latency. Experimental evaluation over RNS-CKKS encrypted CIFAR datasets shows that AutoFHE accelerates secure inference by $1.32 imes$ to $1.8 imes$ compared to methods employing high-degree polynomials. It also improves accuracy by up to 2.56% compared to methods using low-degree polynomials. Lastly, AutoFHE accelerates inference and improves accuracy by $103 imes$ and 3.46%, respectively, compared to CNNs under TFHE.

Motivation & Objective

• To address the inflexibility of manual, architecture-specific polynomial approximations for FHE-secure CNN inference.
• To overcome the suboptimal approximation of individual activation functions instead of the full network function.
• To resolve the trade-off between high-degree (accurate but slow) and low-degree (fast but inaccurate) polynomial approximations.
• To enable automated, scalable, and flexible adaptation of any CNN for secure inference under RNS-CKKS.
• To jointly optimize polynomial degree per layer and bootstrapping placement for Pareto-optimal accuracy-latency trade-offs.

Proposed method

• AutoFHE formulates the adaptation problem as a multi-objective optimization over a custom search space of layerwise mixed-degree polynomial activations.
• It employs a multi-objective search algorithm (MOS) to explore trade-offs between model accuracy and number of bootstrapping operations.
• A composite polynomial coefficient optimization method (R-CCDE) is proposed to refine polynomial coefficients for improved approximation accuracy.
• A polynomial-aware training strategy (PAT) enables end-to-end training of the adapted CNN with mixed-degree polynomial activations.
• The framework jointly optimizes polynomial degree per layer and bootstrapping placement to minimize latency while preserving accuracy under RNS-CKKS.
• It supports any standard CNN architecture and generates a diverse set of solutions spanning the accuracy-latency Pareto frontier.

Experimental results

Research questions

• RQ1 Can an automated method jointly optimize polynomial approximation degree and bootstrapping placement to improve FHE inference efficiency?
• RQ2 Does layerwise mixed-degree polynomial approximation outperform uniform low- or high-degree approximations in accuracy and latency?
• RQ3 Can the framework be flexibly applied to arbitrary CNN architectures without manual re-design?
• RQ4 How does AutoFHE compare to high-degree polynomial methods in inference speed and accuracy?
• RQ5 To what extent does AutoFHE improve upon TFHE-based approaches in latency and accuracy?

Key findings

• AutoFHE accelerates secure inference by 1.32× to 1.8× compared to high-degree polynomial methods under RNS-CKKS.
• It improves top-1 accuracy by up to 2.56% compared to low-degree polynomial methods such as AESPA.
• AutoFHE achieves a 103× speedup in inference latency and 3.46% higher accuracy compared to models evaluated under TFHE (REDsec).
• The framework generates a diverse set of Pareto-optimal solutions that span the accuracy-latency trade-off for any given CNN architecture.
• The joint optimization of mixed-degree polynomials and bootstrapping placement leads to superior performance compared to manually designed or uniformly degree-optimized approaches.
• The proposed multi-objective search and training pipeline (MOS, R-CCDE, PAT) effectively navigates the large search space of FHE-adapted CNNs.