[论文解读] Privacy-Preserving Training of Tree Ensembles over Continuous Data

Samuel S. Adams, Chaitali Choudhary+6 more|arXiv (Cornell University)|Jun 5, 2021

Cryptography and Data Security参考文献 42被引用 5

一句话总结

本文提出了一种基于安全多方计算（MPC）的隐私保护协议，用于在连续数据上训练树集成模型。该协议使两个参与方可协作训练决策树，而无需暴露其原始数据，通过秘密共享和同态加密安全计算最小值/最大值及分裂决策，实现了高效且具备可证明安全保证的私有模型训练。

ABSTRACT

Most existing Secure Multi-Party Computation (MPC) protocols for privacy-preserving training of decision trees over distributed data assume that the features are categorical. In real-life applications, features are often numerical. The standard ``in the clear'' algorithm to grow decision trees on data with continuous values requires sorting of training examples for each feature in the quest for an optimal cut-point in the range of feature values in each node. Sorting is an expensive operation in MPC, hence finding secure protocols that avoid such an expensive step is a relevant problem in privacy-preserving machine learning. In this paper we propose three more efficient alternatives for secure training of decision tree based models on data with continuous features, namely: (1) secure discretization of the data, followed by secure training of a decision tree over the discretized data; (2) secure discretization of the data, followed by secure training of a random forest over the discretized data; and (3) secure training of extremely randomized trees (``extra-trees'') on the original data. Approaches (2) and (3) both involve randomizing feature choices. In addition, in approach (3) cut-points are chosen randomly as well, thereby alleviating the need to sort or to discretize the data up front. We implemented all proposed solutions in the semi-honest setting with additive secret sharing based MPC. In addition to mathematically proving that all proposed approaches are correct and secure, we experimentally evaluated and compared them in terms of classification accuracy and runtime. We privately train tree ensembles over data sets with 1000s of instances or features in a few minutes, with accuracies that are at par with those obtained in the clear. This makes our solution orders of magnitude more efficient than the existing approaches, which are based on oblivious sorting.

研究动机与目标

• 实现对连续数据的树集成模型的安全协作训练，而无需暴露敏感的输入值。
• 解决在涉及多个不可信参与方的场景下，训练决策树过程中保护数据隐私的挑战。
• 设计一种高效协议，利用秘密共享和同态加密安全计算最小值/最大值及分裂条件。
• 确保在处理大规模数据时，双方均无法获知对方的原始数据，同时保持计算效率。
• 为分布式环境中的隐私保护机器学习提供可证明安全且通信高效的解决方案。

提出的方法

• 协议使用秘密共享表示数据值，确保任一参与方均无法推断出原始输入。
• 通过两方协议执行大于等于比较检查，安全计算数据点之间的最小值和最大值。
• 对于每个新数据点，协议基于比较结果的安全条件更新，实现当前最小值和最大值的更新。
• 协议采用转换函数 π₂toQ，将比较结果映射为可用于后续计算的秘密共享值。
• 利用同态加密原语在加密数据上评估比较操作（例如 ≥），在整个过程中保持隐私。
• 算法迭代处理每个数据点，持续维护运行中最小值和最大值的秘密共享表示。

实验结果

研究问题

• RQ1 如何在不暴露各自输入的前提下，由两方安全计算连续数据的最小值和最大值？
• RQ2 在共享数据上，如何实现高效且隐私保护的决策树分裂点确定方法？
• RQ3 能否利用安全多方计算在连续特征上训练树集成模型，同时最小化通信和计算开销？
• RQ4 在分布式环境中，哪些密码学技术可实现安全比较与条件更新？
• RQ5 该协议如何确保在模型训练过程中，任一参与方均无法获取超出必要限度的对方数据信息？

主要发现

• 该协议通过秘密共享和安全比较协议，实现了对连续数据上树集成模型的安全协作训练。
• 通过迭代安全更新实现最小值和最大值的高效计算，减少了对昂贵密码学操作的依赖。
• 使用 π₂toQ 和 π_GEQ 确保比较结果以正确方式编码为秘密共享形式，便于后续处理。
• 通过确保任一参与方在任何阶段均无法获知对方原始数据，有效维护了隐私。
• 该方法通信效率高，适用于大规模数据，其算法的迭代与模块化设计已得到验证。
• 在安全多方计算的标准假设下，该方法提供了可证明的安全保障。