[Paper Review] AsymML: An Asymmetric Decomposition Framework for Privacy-Preserving DNN Training and Inference

Yue Niu, Ramy E. Ali+1 more|arXiv (Cornell University)|Oct 4, 2021

Privacy-Preserving Technologies in Data25 references4 citations

TL;DR

AsymML proposes an asymmetric decomposition framework that splits deep neural network models into trusted (privacy-sensitive) and untrusted (computationally intensive) parts, offloading the former to trusted execution environments (TEEs) and the latter to untrusted GPUs. This enables 11.2× faster inference and 7.6× faster training compared to TEE-only execution, preserving privacy while significantly improving performance by exploiting low-rank characteristics in features.

ABSTRACT

Leveraging parallel hardware (e.g. GPUs) to conduct deep neural network (DNN) training/inference, though significantly speeds up the computations, raises several data privacy concerns. Trusted execution environments (TEEs) have emerged as a promising solution to enable privacy-preserving inference and training. TEEs, however, have limited memory and computation resources which renders it not comparable to untrusted parallel hardware in performance. To mitigate the trade-off between privacy and computing performance, we propose an asymmetric model decomposition framework, AsymML, to (1) accelerate training/inference using parallel hardware; and (2) preserve privacy using TEEs. By exploiting the low-rank characteristics in data and intermediate features, AsymML asymmetrically splits a DNN model into trusted and untrusted parts: the trusted part features privacy-sensitive data but incurs small compute/memory costs; while the untrusted part is computationally-intensive but not privacy-sensitive. Computing performance and privacy are guaranteed by respectively delegating the trusted and untrusted part to TEEs and GPUs. Furthermore, we present a theoretical rank bound analysis showing that low-rank characteristics are still preserved in intermediate features, which guarantees efficiency of AsymML. Extensive evaluations on DNN models shows that AsymML delivers $11.2 imes$ speedup in inference, $7.6 imes$ in training compared to the TEE-only executions.

Motivation & Objective

• Address the performance bottleneck in privacy-preserving DNN training and inference when relying solely on trusted execution environments (TEEs), which have limited memory and compute.
• Overcome the trade-off between privacy and computational efficiency by asymmetrically partitioning DNN models into trusted and untrusted components.
• Leverage low-rank properties in data and intermediate features to ensure that the trusted component remains small and efficient while maintaining model accuracy.
• Enable high-performance DNN inference and training by delegating the untrusted, compute-heavy parts to untrusted parallel hardware (e.g., GPUs), while preserving data privacy via TEEs.
• Theoretically and empirically validate that low-rank characteristics are preserved in intermediate features, enabling efficient and secure model decomposition.

Proposed method

• Asymmetrically decompose a DNN into two parts: a trusted component stored and executed in a TEE, and an untrusted component executed on untrusted parallel hardware (e.g., GPUs).
• Exploit low-rank characteristics in input data and intermediate feature maps to minimize the size and computational cost of the trusted component.
• Formulate the decomposition such that privacy-sensitive weights and activations are confined to the TEE, while the bulk of the computation is offloaded to untrusted hardware.
• Apply theoretical rank bound analysis to prove that intermediate features in deep networks maintain low-rank structure, ensuring the feasibility and efficiency of the decomposition.
• Design a training and inference pipeline that synchronizes computations between the TEE and untrusted hardware, minimizing communication overhead and preserving model accuracy.
• Use a hybrid execution model where the trusted part performs privacy-critical operations (e.g., sensitive weight updates), and the untrusted part accelerates bulk matrix operations.

Experimental results

Research questions

• RQ1 Can low-rank characteristics in data and intermediate features be leveraged to enable efficient and secure model decomposition for privacy-preserving DNNs?
• RQ2 To what extent can performance be improved by offloading computationally intensive parts of DNNs to untrusted GPUs while keeping privacy-sensitive components in TEEs?
• RQ3 Does the low-rank structure of intermediate features remain preserved after multiple layers, enabling effective decomposition without significant accuracy loss?
• RQ4 What is the theoretical bound on the rank of intermediate features in deep networks, and how does it support the design of an efficient asymmetric decomposition framework?
• RQ5 How do the performance gains of AsymML compare to TEE-only execution in terms of inference and training speed?

Key findings

• AsymML achieves 11.2× speedup in DNN inference compared to TEE-only execution by offloading the bulk of computation to untrusted GPUs.
• The framework delivers 7.6× speedup in DNN training relative to TEE-only training, significantly improving performance without compromising privacy.
• Theoretical analysis confirms that low-rank characteristics are preserved in intermediate features, justifying the efficiency of the asymmetric decomposition approach.
• The trusted component in AsymML incurs minimal computational and memory overhead, making it feasible to run within the constrained resources of a TEE.
• Empirical evaluations on DNN models demonstrate that AsymML maintains high model accuracy while drastically reducing execution time on privacy-sensitive workloads.
• The framework effectively decouples privacy and performance concerns by isolating sensitive operations in TEEs and accelerating non-sensitive operations on untrusted hardware.