[Paper Review] Privacy-Preserving Tree-Based Inference with TFHE

Jordan Fréry, Andrei Stoian+5 more|arXiv (Cornell University)|Feb 13, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper presents a privacy-preserving inference framework for tree-based machine learning models using the TFHE fully homomorphic encryption scheme. By leveraging optimized integer representations and programmable bootstrapping, it achieves near-identical accuracy to clear-data inference with competitive latency, enabling secure deployment of decision trees, random forests, and gradient boosted trees on encrypted data.

ABSTRACT

Privacy enhancing technologies (PETs) have been proposed as a way to protect the privacy of data while still allowing for data analysis. In this work, we focus on Fully Homomorphic Encryption (FHE), a powerful tool that allows for arbitrary computations to be performed on encrypted data. FHE has received lots of attention in the past few years and has reached realistic execution times and correctness. More precisely, we explain in this paper how we apply FHE to tree-based models and get state-of-the-art solutions over encrypted tabular data. We show that our method is applicable to a wide range of tree-based models, including decision trees, random forests, and gradient boosted trees, and has been implemented within the Concrete-ML library, which is open-source at https://github.com/zama-ai/concrete-ml. With a selected set of use-cases, we demonstrate that our FHE version is very close to the unprotected version in terms of accuracy.

Motivation & Objective

• To enable secure inference of tree-based models on encrypted data without exposing sensitive inputs.
• To address the challenge of high latency and accuracy loss in existing FHE-based tree inference methods.
• To support both single decision trees and ensemble models (random forests, XGBoost) with minimal accuracy degradation.
• To automate crypto-system parameter selection for optimal trade-offs between precision, speed, and error tolerance.
• To demonstrate that FHE-based inference can achieve accuracy within 1% of clear-data performance while remaining practical for real-world deployment.

Proposed method

• The method uses TFHE's programmable bootstrapping (PBS) to evaluate non-linear decision functions (e.g., comparisons) homomorphically, enabling correct tree traversal on encrypted data.
• It represents features as quantized integers to control message space size and reduce noise growth during homomorphic operations.
• Leveled homomorphic computation is enabled by parameterizing the FHE scheme to support accumulation of ciphertexts over multiple tree nodes.
• An automated optimization pipeline determines optimal FHE parameters (e.g., security level, precision) based on model structure and desired error tolerance.
• The approach supports both classification and regression tasks across decision trees, random forests, and gradient boosted trees.
• The implementation is integrated into the open-source Concrete-ML library, enabling end-to-end privacy-preserving inference.

Figure 1: TFHE operations representing algorithm 2 . The $\Sigma$ nodes represent leveled accumulation and $T[x]$ nodes represent table look-up operations implemented with PBS

Experimental results

Research questions

• RQ1 Can TFHE-based homomorphic encryption enable accurate and efficient inference for tree-based models on encrypted tabular data?
• RQ2 How does the use of programmable bootstrapping improve the feasibility of evaluating non-linear decision functions in FHE?
• RQ3 What is the trade-off between quantization precision, FHE execution latency, and model accuracy in privacy-preserving inference?
• RQ4 Can the method achieve inference accuracy comparable to clear-data execution while maintaining strong security guarantees?
• RQ5 How does the choice of PBS error probability affect performance and correctness in real-world deployments?

Key findings

• FHE-based inference achieved accuracy within 1% of clear-data inference across all tested datasets, including adult, wine, steel, and heart disease.
• Latency for FHE inference ranged from 1.18s (wdbc) to 81.94s (heart-h), with a median speedup ratio of ~10,000x compared to FP32 models.
• Using a PBS error probability of 0.05 reduced latency by up to 8x while preserving model accuracy, demonstrating a viable trade-off for production use.
• The method achieved 98.5% accuracy on the wine dataset with a random forest, matching the 98.1% accuracy of the clear-data model.
• For the steel dataset, FHE-XGBoost achieved 100% accuracy, matching the FP32 baseline, with a latency of 8.78s.
• The automated parameter selection process enabled optimal configuration of FHE parameters for each model, minimizing manual tuning.

Figure 2: Experiment reporting the f1-score and average precision with varying precision on the spambase dataset.