[Paper Review] Towards resilient machine learning for ransomware detection

Li Chen, Chih-Yuan Yang+2 more|arXiv (Cornell University)|Dec 21, 2018

Advanced Malware Detection TechniquesComputer Science50 references21 citations

TL;DR

This paper proposes a GAN-based framework to generate adversarial dynamic features that evade machine learning-based ransomware detectors by mimicking malicious behaviors while appearing benign. Despite high accuracy in real-world detection, six of seven tested classifiers failed to detect the GAN-generated samples, revealing critical vulnerabilities in ML resilience for ransomware defense.

ABSTRACT

There has been a surge of interest in using machine learning (ML) to automatically detect malware through their dynamic behaviors. These approaches have achieved significant improvement in detection rates and lower false positive rates at large scale compared with traditional malware analysis methods. ML in threat detection has demonstrated to be a good cop to guard platform security. However it is imperative to evaluate - is ML-powered security resilient enough? In this paper, we juxtapose the resiliency and trustworthiness of ML algorithms for security, via a case study of evaluating the resiliency of ransomware detection via the generative adversarial network (GAN). In this case study, we propose to use GAN to automatically produce dynamic features that exhibit generalized malicious behaviors that can reduce the efficacy of black-box ransomware classifiers. We examine the quality of the GAN-generated samples by comparing the statistical similarity of these samples to real ransomware and benign software. Further we investigate the latent subspace where the GAN-generated samples lie and explore reasons why such samples cause a certain class of ransomware classifiers to degrade in performance. Our focus is to emphasize necessary defense improvement in ML-based approaches for ransomware detection before deployment in the wild. Our results and discoveries should pose relevant questions for defenders such as how ML models can be made more resilient for robust enforcement of security objectives.

Motivation & Objective

• To evaluate the resilience of machine learning-based ransomware detection systems against adversarial attacks.
• To develop a GAN-based framework that automatically generates dynamic features mimicking real ransomware behaviors.
• To quantify the maliciousness and statistical similarity of generated samples to real ransomware and benign software.
• To identify blind spots in existing classifiers by analyzing latent feature spaces where adversarial samples lie.
• To propose adversarial training and quality metrics to improve model robustness before deployment in production environments.

Proposed method

• Train a generative adversarial network (GAN) on real dynamic system logs from ransomware and benign applications to generate adversarial feature sequences.
• Use statistical similarity metrics (e.g., K-S test, Wasserstein distance) to compare GAN-generated logs with real benign and malicious logs.
• Map generated samples into latent feature spaces (e.g., Text-CNN embeddings) to analyze classifier vulnerabilities.
• Develop a ransomware simulation program that maps GAN-generated feature sequences into actual I/O operations (e.g., file rename, high-entropy write) to produce real malicious behavior.
• Apply adversarial quality metrics to ensure generated samples retain malicious intent while evading detection.
• Use GAN-generated samples for adversarial training to improve the robustness of existing ransomware classifiers.

Experimental results

Research questions

• RQ1 Can GAN-generated dynamic features successfully evade state-of-the-art ML-based ransomware detectors?
• RQ2 How similar are the GAN-generated samples to real ransomware and benign software in terms of statistical distribution?
• RQ3 In which latent feature subspaces do adversarial samples lie, and why do they cause classifier degradation?
• RQ4 What adversarial quality metrics can reliably assess the maliciousness and evasiveness of generated samples?
• RQ5 Can adversarial training using GAN-generated samples improve the resilience of ransomware classifiers in real-world deployment?

Key findings

• Six out of seven highly effective ransomware classifiers failed to detect the majority of GAN-generated adversarial samples, despite high accuracy on real data.
• GAN-generated samples showed high statistical similarity to real ransomware in the original feature space but were closer to benign logs in the Text-CNN latent space, indicating they lie in a blind spot of the classifier.
• The Text-CNN classifier was particularly vulnerable, as adversarial samples were statistically closer to benign logs in its latent space, explaining its failure to detect them.
• The SVM-radial and XGB-Text-CNN ensembles showed higher robustness, suggesting that multi-classifier ensembles can improve defense-in-depth against adversarial probing.
• The proposed adversarial quality metrics effectively quantified maliciousness and similarity, enabling reliable assessment of generated samples.
• The framework successfully generated real I/O sequences that bypassed ML detection while maintaining full ransomware functionality, demonstrating practical evasion capability.