[Paper Review] A Survey of Machine Learning Methods and Challenges for Windows Malware Classification

Edward Raff, Charles Nicholas|arXiv (Cornell University)|Jun 15, 2020

Advanced Malware Detection TechniquesComputer Science225 references37 citations

TL;DR

This survey reviews how machine learning has been applied to Windows PE malware classification, detailing data collection, feature extraction, modeling approaches, evaluation, and key practical challenges.

ABSTRACT

Malware classification is a difficult problem, to which machine learning methods have been applied for decades. Yet progress has often been slow, in part due to a number of unique difficulties with the task that occur through all stages of the developing a machine learning system: data collection, labeling, feature creation and selection, model selection, and evaluation. In this survey we will review a number of the current methods and challenges related to malware classification, including data collection, feature extraction, and model construction, and evaluation. Our discussion will include thoughts on the constraints that must be considered for machine learning based solutions in this domain, and yet to be tackled problems for which machine learning could also provide a solution. This survey aims to be useful both to cybersecurity practitioners who wish to learn more about how machine learning can be applied to the malware problem, and to give data scientists the necessary background into the challenges in this uniquely complicated space.

Motivation & Objective

• Identify the end-to-end ML pipeline for Windows malware classification, from data collection to evaluation.
• Summarize static and dynamic feature extraction methods and their trade-offs.
• Survey common ML models used on feature representations (vectors, sequences, graphs) and their applicability.
• Discuss practical constraints, data labeling, and reproducibility issues impacting real-world deployment.
• Highlight open research questions and areas needing further investigation.

Proposed method

• Classify features into static and dynamic categories and summarize representative representations (fixed-length, sequence, graph).
• Describe dynamic feature extraction methods that track API calls, system changes, and network activity.
• Discuss static feature extraction from raw bytes, entropy, PE headers, and disassembly outputs.
• Outline modeling approaches spanning standard vector-based methods, sequence models, and graph-based techniques.
• Discuss evaluation challenges and the lack of standard datasets and labeling biases.
• Address data collection, labeling costs, and concept drift as fundamental obstacles.

Experimental results

Research questions

• RQ1 What are the main data collection and labeling challenges in Windows malware ML pipelines?
• RQ2 How do static, dynamic, and graph-based features compare for Windows PE malware classification?
• RQ3 What ML models are most suitable for different feature representations (vectors, sequences, graphs) in this domain?
• RQ4 What are the key evaluation and reproducibility issues hindering cross-study comparisons?
• RQ5 What future directions and open problems most impact practical malware classification systems?

Key findings

• Data collection and labeling are major bottlenecks, with biases from honeypots, VirusTotal thresholds, and lack of standard datasets.
• Static vs dynamic features each have strengths and weaknesses, including obfuscation and VM-detection issues in dynamic analysis.
• Concept drift and adversarial evasion pose ongoing challenges to model generalization over time.
• Lack of standard datasets (historical reliance on Windows-installed binaries) hinders cross-study comparisons and reproducibility.
• AV-based labeling introduces biases and may not reflect hard cases near decision boundaries.
• Graph-based representations, while powerful, are underutilized compared to vector and sequence approaches.