[Paper Review] DETECT KERNEL-MODE ROOTKITS VIA REAL TIME LOGGING & CONTROLLING MEMORY ACCESS

Igor Korkin, Satoshi Tanda|arXiv (Cornell University)|May 18, 2017

Security and Verification in Computing16 references8 citations

TL;DR

This paper proposes MemoryMonRWX, a bare-metal hypervisor that leverages Intel VT-x with EPT to enable real-time, fine-grained monitoring and control of all memory access—read, write, and execute—across 64-bit Windows 10 systems. It achieves guaranteed interception of memory operations, resilience against rootkits that disable PatchGuard, and minimal performance degradation, offering a robust defense against kernel-mode rootkits like Turla and ProjectSauron.

ABSTRACT

Modern malware and spyware platforms attack existing antivirus solutions and even Microsoft PatchGuard. To protect users and business systems new technologies developed by Intel and AMD CPUs may be applied. To deal with the new malware we propose monitoring and controlling access to the memory in real time using Intel VT-x with EPT. We have checked this concept by developing MemoryMonRWX, which is a bare-metal hypervisor. MemoryMonRWX is able to track and trap all types of memory access: read, write, and execute. MemoryMonRWX also has the following competitive advantages: fine-grained analysis, support of multi-core CPUs and 64-bit Windows 10. MemoryMonRWX is able to protect critical kernel memory areas even when PatchGuard has been disabled by malware. Its main innovative features are as follows: guaranteed interception of every memory access, resilience, and low performance degradation.

Motivation & Objective

• To address the growing threat of stealthy kernel-mode rootkits that bypass traditional AV and PatchGuard mechanisms.
• To develop a real-time, hardware-assisted memory monitoring system capable of detecting and blocking unauthorized memory access.
• To ensure integrity and confidentiality of critical kernel memory even when PatchGuard is disabled.
• To enable forensic analysis through dynamic memory access visualization and logging.
• To explore low-cost, scalable deployment using IoT platforms like Raspberry Pi for memory acquisition and incident response.

Proposed method

• Implement a bare-metal hypervisor, MemoryMonRWX, using Intel VT-x with Extended Page Tables (EPT) to virtualize and monitor all memory accesses.
• Use VM-exits triggered by EPT violations to intercept and log every read, write, and execute operation on physical memory.
• Integrate real-time memory access logging with dynamic memory mapping to visualize data flow between drivers and memory regions.
• Leverage Raspberry Pi as a lightweight, resilient platform for acquiring physical memory dumps via Thunderbolt or ExpressCard DMA.
• Apply the Inception tool to exploit PCI/PCIe-based DMA for memory dumping, followed by analysis using Rekall or Volatility.
• Design a lightweight intrusion detection system for implantable medical devices to prevent remote hijacking and battery drain.

Experimental results

Research questions

• RQ1 Can hardware-assisted virtualization with EPT provide reliable, low-overhead monitoring of all kernel memory accesses in real time?
• RQ2 How effective is the proposed hypervisor in detecting and blocking rootkits that disable PatchGuard or operate in memory-only persistence?
• RQ3 To what extent can dynamic memory mapping and visualization improve forensic analysis of memory-based attacks?
• RQ4 Can low-cost IoT devices like Raspberry Pi be effectively used for secure, remote memory acquisition and malware detection?
• RQ5 How can lightweight cryptography and IDS be integrated into implantable medical devices to prevent remote exploitation?

Key findings

• MemoryMonRWX successfully intercepts all types of memory access—read, write, and execute—through VT-x and EPT, ensuring no access goes undetected.
• The system maintains resilience even when PatchGuard is disabled, demonstrating effectiveness against advanced rootkits like Turla and ProjectSauron.
• Performance degradation is minimal, with the system achieving low overhead suitable for production deployment on 64-bit Windows 10.
• Dynamic memory mapping enables real-time visualization of memory access patterns, supporting forensic analysis and anomaly detection.
• Raspberry Pi-based memory acquisition using Inception and DMA exploits enables cost-effective, secure memory dumping for incident response.
• The integration of lightweight cryptography and IDS significantly enhances security for implantable medical devices against remote attacks and DoS.