[Paper Review] Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices

Peter M. VanNostrand, Ioannis Kyriazis+3 more|arXiv (Cornell University)|Aug 28, 2019

Privacy-Preserving Technologies in Data12 references18 citations

TL;DR

This paper proposes using ARM TrustZone to confidentially execute proprietary deep learning models on untrusted mobile devices, leveraging a trusted execution environment to protect model weights and inference logic. It introduces a layer-based model partitioning strategy that runs critical components in the secure world while minimizing performance overhead, demonstrating a viable path for securing intellectual property in edge AI deployments.

ABSTRACT

Performing deep learning on end-user devices provides fast offline inference results and can help protect the user's privacy. However, running models on untrusted client devices reveals model information which may be proprietary, i.e., the operating system or other applications on end-user devices may be manipulated to copy and redistribute this information, infringing on the model provider's intellectual property. We propose the use of ARM TrustZone, a hardware-based security feature present in most phones, to confidentially run a proprietary model on an untrusted end-user device. We explore the limitations and design challenges of using TrustZone and examine potential approaches for confidential deep learning within this environment. Of particular interest is providing robust protection of proprietary model information while minimizing total performance overhead.

Motivation & Objective

• To address the confidentiality of proprietary deep learning models when executed on untrusted end-user devices.
• To explore the feasibility and performance trade-offs of running DNN inference within ARM TrustZone's trusted execution environment (TEE).
• To design a model partitioning strategy that balances security, performance, and memory constraints in TEEs.
• To identify and mitigate side-channel vulnerabilities in TEE-based deep learning execution.
• To establish a foundation for secure, production-ready confidential inference on mobile devices.

Proposed method

• Utilizes ARM TrustZone to create a hardware-isolated secure world for confidential model execution, separating it from the untrusted normal world OS.
• Employs OP-TEE as the TEE implementation, enforcing code integrity via RSA-signed trusted applications and secure memory isolation.
• Proposes a generalized layer-based partitioning approach, allowing individual layers of a DNN to be executed in the secure world, unlike prior two-part partitions.
• Introduces branched execution partitioning inspired by Sutton et al., enabling non-uniform layer distribution to optimize performance and memory usage.
• Relies on remote attestation to verify the integrity of the trusted application and secure boot to establish a root of trust.
• Designs for data-oblivious execution to defend against side-channel leaks such as memory access patterns and timing variations.

Experimental results

Research questions

• RQ1 How can proprietary deep learning models be securely executed on untrusted mobile devices without exposing model weights or architecture?
• RQ2 What are the performance and engineering trade-offs of running DNN inference inside ARM TrustZone’s TEE with limited secure memory?
• RQ3 How can model partitioning be generalized beyond two-part splits to improve performance and memory utilization in TEEs?
• RQ4 What side-channel vulnerabilities exist in TEE-based deep learning, and how can they be mitigated through algorithmic hardening?
• RQ5 What system-level mechanisms (e.g., key management, remote attestation) are required to enable secure, scalable deployment of confidential models?

Key findings

• Layer-based partitioning enables fine-grained control over which model components run in the secure world, improving memory efficiency and performance over monolithic or two-part partitioning.
• The use of branched execution partitioning allows for better load balancing and reduced latency by avoiding full model offloading to the untrusted environment.
• Significant performance overhead is inherent in TEE execution due to isolated memory and limited system call support, but partitioning strategies can mitigate this impact.
• Side-channel attacks remain a critical threat, particularly through observable execution patterns such as memory access and timing, necessitating data-oblivious algorithm design.
• Remote attestation and secure key distribution are essential for verifying trusted application integrity and protecting model confidentiality at scale.
• A hardware root of trust and secure boot are required to prevent tampering of the TEE OS, highlighting the need for collaboration with device manufacturers.