[论文解读] Efficient Batch Homomorphic Encryption for Vertically Federated XGBoost

Wuxing Xu, Hao Fan+2 more|arXiv (Cornell University)|Dec 8, 2021

Privacy-Preserving Technologies in Data被引用 10

一句话总结

该论文提出了一种高效的批量同态加密方法，用于垂直联邦XGBoost，将一阶和二阶梯度编码为单一数值，使同态加密和密文传输成本几乎减半。与SecureBoost相比，该方法使树构建速度提升高达50%，总运行时间减少30%以上，且在AUC和KS评分上精度损失可忽略不计。

ABSTRACT

More and more orgainizations and institutions make efforts on using external data to improve the performance of AI services. To address the data privacy and security concerns, federated learning has attracted increasing attention from both academia and industry to securely construct AI models across multiple isolated data providers. In this paper, we studied the efficiency problem of adapting widely used XGBoost model in real-world applications to vertical federated learning setting. State-of-the-art vertical federated XGBoost frameworks requires large number of encryption operations and ciphertext transmissions, which makes the model training much less efficient than training XGBoost models locally. To bridge this gap, we proposed a novel batch homomorphic encryption method to cut the cost of encryption-related computation and transmission in nearly half. This is achieved by encoding the first-order derivative and the second-order derivative into a single number for encryption, ciphertext transmission, and homomorphic addition operations. The sum of multiple first-order derivatives and second-order derivatives can be simultaneously decoded from the sum of encoded values. We are motivated by the batch idea in the work of BatchCrypt for horizontal federated learning, and design a novel batch method to address the limitations of allowing quite few number of negative numbers. The encode procedure of the proposed batch method consists of four steps, including shifting, truncating, quantizing and batching, while the decoding procedure consists of de-quantization and shifting back. The advantages of our method are demonstrated through theoretical analysis and extensive numerical experiments.

研究动机与目标

• 解决垂直联邦XGBoost训练中同态加密带来的高计算与通信开销问题。
• 减少安全梯度聚合过程中所需的加密操作次数和密文传输次数。
• 实现在具有垂直划分数据的机构之间高效、隐私保护的XGBoost训练。
• 克服先前批量方法在梯度编码中难以处理负数的局限性。
• 在显著提升效率的同时，保持与SecureBoost等最先进方法相当的模型精度。

提出的方法

• 通过四步法（移位、截断、量化、批量处理）将一阶导数 $g_i$ 和二阶导数 $h_i$ 编码为单个数值。
• 对编码后的单一数值执行同态加密和加法运算，实现在一次操作中同时计算聚合后的 $g$ 和 $h$。
• 通过去量化和反向移位解码编码值的和，恢复出各个 $g$ 和 $h$ 值。
• 引入量化参数 $\alpha_{\sf{max}}$ 和缩放因子 $r$，以控制精度并防止编码过程中的溢出。
• 设计批量方案以安全处理负数，避免先前方法（如BatchCrypt）常见的溢出问题。
• 在模型训练期间使用Paillier同态加密，对编码梯度执行安全的加法运算。

实验结果

研究问题

• RQ1 能否通过批量处理一阶和二阶梯度，减少垂直联邦XGBoost中同态加密操作和密文传输的次数？
• RQ2 如何设计批量方法以在不发生溢出的情况下处理负数，而现有方法则存在此问题？
• RQ3 量化参数对梯度编码中精度损失和数值稳定性有何影响？
• RQ4 与SecureBoost相比，该方法在训练效率和模型精度方面表现如何？
• RQ5 尽管存在编码引起的量化影响，该批量方法是否仍能保持模型性能（以AUC和KS衡量）？

主要发现

• 所提方法在大规模数据集上使垂直联邦XGBoost训练的总运行时间减少30%以上，与SecureBoost相比，树构建时间最多减少50%。
• 对于91,500个样本的数据集，总运行时间减少至SecureBoost的75%，且随着问题规模增大，性能优势进一步提升。
• 该方法保持了相近的模型精度，所有评估数据集和配置下AUC和KS值差异均小于0.005。
• 随着提升轮数的增加，效率增益也提升，在90棵树时，每棵树训练速度最快可提升50%。
• 通过合理选择 $\alpha_{\sf{max}}$ 和 $r$，可实现接近最优的性能，最大限度减少溢出并保持精度。
• 理论分析证实，该方法避免了因负数相加导致的溢出问题，这是先前批量技术的关键局限。