[Paper Review] East: Efficient and Accurate Secure Transformer Framework for Inference

Yuanchao Ding, Hua Guo+5 more|arXiv (Cornell University)|Aug 19, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

East proposes a secure, efficient, and accurate framework for privacy-preserving Transformer inference using a novel oblivious piecewise polynomial evaluation (OPPE) algorithm to accelerate non-linear functions like GELU and tanh. It achieves 1.5× lower runtime and 2.6× lower communication than prior arts, maintaining plaintext-level accuracy without model retraining.

ABSTRACT

Transformer has been successfully used in practical applications, such as ChatGPT, due to its powerful advantages. However, users' input is leaked to the model provider during the service. With people's attention to privacy, privacy-preserving Transformer inference is on the demand of such services. Secure protocols for non-linear functions are crucial in privacy-preserving Transformer inference, which are not well studied. Thus, designing practical secure protocols for non-linear functions is hard but significant to model performance. In this work, we propose a framework \emph{East} to enable efficient and accurate secure Transformer inference. Firstly, we propose a new oblivious piecewise polynomial evaluation algorithm and apply it to the activation functions, which reduces the runtime and communication of GELU by over 1.5$ imes$ and 2.5$ imes$, compared to prior arts. Secondly, the secure protocols for softmax and layer normalization are carefully designed to faithfully maintain the desired functionality. Thirdly, several optimizations are conducted in detail to enhance the overall efficiency. We applied \emph{East} to BERT and the results show that the inference accuracy remains consistent with the plaintext inference without fine-tuning. Compared to Iron, we achieve about 1.8$ imes$ lower communication within 1.2$ imes$ lower runtime.

Motivation & Objective

• Address the lack of efficient and accurate secure protocols for non-linear functions in privacy-preserving Transformer inference.
• Enable secure inference that preserves both client input privacy and server model parameter privacy.
• Maintain the original model architecture without requiring retraining or architectural modifications.
• Reduce the performance overhead of secure non-linear functions, which previously accounted for up to 87% of communication costs.
• Design optimized, MPC-friendly protocols for GELU, tanh, softmax, and layer normalization with minimal error leakage.

Proposed method

• Propose a new oblivious piecewise polynomial evaluation (OPPE) algorithm to securely evaluate non-linear functions like GELU and tanh using piecewise polynomial approximation.
• Design secure protocols for softmax and layer normalization using error-limited conversion methods and secret sharing to ensure correctness and privacy.
• Optimize offline and online phases by reducing Beaver triple usage and leveraging efficient precomputation for improved communication and runtime.
• Integrate the protocols into a full inference pipeline for BERT, preserving the original model structure and avoiding retraining.
• Use homomorphic encryption (HE) and secret sharing (SS) as the underlying cryptographic primitives to ensure security and correctness.
• Apply dimension-specific optimizations for functions like GELU (d=3, m=8) and tanh (d=3, m=7) to balance accuracy and efficiency.

Experimental results

Research questions

• RQ1 How can non-linear functions like GELU and tanh be securely evaluated in a privacy-preserving Transformer inference setting with minimal performance overhead?
• RQ2 Can a novel oblivious evaluation method reduce communication and runtime costs for non-linear functions compared to lookup-table-based approaches?
• RQ3 To what extent can secure softmax and layer normalization protocols be designed to maintain accuracy and efficiency in MPC-based inference?
• RQ4 Can the original model architecture be preserved in secure inference without retraining, while still achieving performance close to plaintext?
• RQ5 How does the proposed framework compare to state-of-the-art solutions like Iron in terms of runtime, communication, and inference accuracy?

Key findings

• The proposed OPPE algorithm reduces GELU's communication cost by 2.6× and runtime by 1.5× compared to NFGen, a prior state-of-the-art method.
• For GELU, East achieves 1.5× lower runtime and 2.9× lower communication than Iron, the current SOTA in secure Transformer inference.
• The secure softmax and layer normalization protocols scale linearly with input dimension, requiring ~5 seconds and ~2 seconds of runtime for 1000-dimensional inputs, respectively.
• East maintains inference accuracy identical to plaintext models, with no drop in F1-score, precision, recall, or accuracy across tested benchmarks.
• The framework reduces total communication by 1.8× and runtime by 1.2× compared to Iron, saving ~256 seconds and ~30 GB for BERT-Base inference.
• The offline phase sees up to 3× optimization due to reduced Beaver triple usage, significantly improving precomputation efficiency.