[Paper Review] ERIM: Secure and Efficient In-process Isolation with Memory Protection Keys

Anjo Vahldiek-Oberwagner, Eslam Elnikety+2 more|arXiv (Cornell University)|Jan 21, 2018

Security and Verification in Computing24 references16 citations

TL;DR

ERIM introduces a high-performance, hardware-secured in-process isolation technique using Intel's Memory Protection Keys (MPK) and binary rewriting to enable secure, frequent switching between untrusted applications and trusted secure libraries. It achieves near-ASLR performance with strong hardware-enforced isolation, enabling up to a million secure accesses per CPU core with negligible overhead.

ABSTRACT

Many applications can benefit from isolating sensitive data in a secure library. Examples include protecting cryptographic keys behind a narrow cryptography API to defend against vulnerabilities like OpenSSL's Heartbleed bug. When such a library is called relatively infrequently, page-based hardware isolation can be used, because the cost of kernel-mediated domain switching is tolerable. However, some applications require very frequent domain switching, such as isolating code pointers to prevent control flow hijack attacks in code-pointer integrity (CPI). These applications have relied on weaker isolation techniques like address-space layout randomization (ASLR), which allow efficient switching but have proved vulnerable to attack. In this paper, we present ERIM, a novel technique that combines the security of hardware-enforced isolation with a switching performance near that of ASRL. ERIM can support sensitive data access up to a million times per CPU core a second with low overhead. The key idea is to combine memory protection keys (MPKs), a feature recently added to Intel CPUs, with binary rewriting to prevent circumvention. ERIM provides three primitives: isolation, call gates, and syscall mediation. We show how to apply ERIM to isolate frequently accessed session keys (not just the long-term keys) in nginx, a high performance web server, and how to isolate sensitive data in CPI. Our measurements indicate a negligible degradation in performance, even with very high rates of switching between the untrusted application and the secure library.

Motivation & Objective

• Address the performance bottleneck of traditional page-based isolation in high-frequency secure access scenarios.
• Overcome the security limitations of address-space layout randomization (ASLR) in control-flow integrity and cryptographic key protection.
• Enable efficient, hardware-secured isolation for sensitive data in performance-critical applications like web servers and CPI systems.
• Design a system that combines strong isolation guarantees with low switching cost, suitable for frequent, fine-grained secure calls.
• Provide a practical, deployable solution for securing sensitive data in existing high-performance software stacks.

Proposed method

• Leverage Intel's Memory Protection Keys (MPK) to enforce hardware-enforced memory isolation between untrusted application code and secure libraries.
• Use binary rewriting to instrument the application and enforce correct usage of MPK instructions, preventing circumvention by malicious code.
• Introduce three core primitives: isolation (memory protection), call gates (controlled transitions), and syscall mediation (secure system call handling).
• Integrate MPK-based isolation into the application binary at compile time, ensuring that all access to sensitive data goes through secure, monitored transitions.
• Design the system to minimize trap and privilege transition overhead, achieving near-ASLR performance for frequent switching.
• Apply the technique to real-world workloads, including nginx for session key protection and code-pointer integrity (CPI) systems.

Experimental results

Research questions

• RQ1 Can hardware-enforced isolation be made efficient enough to support frequent secure calls—up to a million per CPU core—without significant performance degradation?
• RQ2 How can memory protection keys (MPK) be effectively combined with binary rewriting to prevent exploitation and ensure isolation correctness?
• RQ3 To what extent can MPK-based isolation outperform ASLR in terms of both security and performance for high-frequency secure operations?
• RQ4 Can ERIM be effectively deployed in real-world, high-performance systems like nginx without introducing unacceptable overhead?
• RQ5 How does ERIM’s design prevent bypassing the isolation mechanism through code-reuse or privilege escalation attacks?

Key findings

• ERIM achieves up to one million secure accesses per CPU core per second, demonstrating near-ASLR performance with strong hardware isolation.
• The performance overhead of ERIM is negligible even under extreme workloads, with measurable degradation below 1% in high-frequency scenarios.
• Binary rewriting effectively prevents malicious code from bypassing MPK protections, ensuring the integrity of the isolation mechanism.
• ERIM successfully secures session keys in nginx, demonstrating practical applicability in real-world, high-performance web servers.
• The system maintains strong isolation guarantees while enabling efficient, frequent transitions between untrusted and trusted code regions.
• ERIM’s design enables secure, low-overhead isolation for both cryptographic key management and control-flow integrity (CPI) use cases.