[Paper Review] Microsoft Malware Classification Challenge

Royi Ronen, Marian Radu+3 more|arXiv (Cornell University)|Feb 22, 2018

Advanced Malware Detection Techniques32 references289 citations

TL;DR

This paper introduces the Microsoft Malware Classification Challenge dataset, a half-terabyte collection of disassembly and bytecode for over 20k malware samples across 9 families, and reviews how the dataset has been used as a benchmark in malware classification research and Kaggle competitions.

ABSTRACT

The Microsoft Malware Classification Challenge was announced in 2015 along with a publication of a huge dataset of nearly 0.5 terabytes, consisting of disassembly and bytecode of more than 20K malware samples. Apart from serving in the Kaggle competition, the dataset has become a standard benchmark for research on modeling malware behaviour. To date, the dataset has been cited in more than 50 research papers. Here we provide a high-level comparison of the publications citing the dataset. The comparison simplifies finding potential research directions in this field and future performance evaluation of the dataset.

Motivation & Objective

• Motivate the need to classify polymorphic malware efficiently at large scale.
• Describe the dataset and its components to enable research and benchmarking.
• Summarize how the dataset has been used in publications and competitions to guide future work.

Proposed method

• Describe the dataset composition including raw hex content and metadata from IDA disassembly.
• Define the 9 malware families and corresponding train sample counts.
• Explain the dataset’s role as a benchmark for modeling malware behavior and for Kaggle competition.
• Provide a high-level literature comparison of works citing the dataset and their focus areas.

Experimental results

Research questions

• RQ1 How large and diverse is the Microsoft malware dataset, and how are samples labeled into 9 families?
• RQ2 What kinds of features and research directions have been explored using this dataset (e.g., feature engineering, scalability, robustness, deep learning)?
• RQ3 How has the dataset influenced subsequent malware classification research and benchmarking in the literature?

Key findings

• The dataset is roughly half a terabyte uncompressed and contains disassembly-derived metadata plus raw binary content for over 20K samples.
• There are 9 malware families in the dataset, each with a specific train sample count and label mapping (e.g., Ramnit 1541, Lollipop 2478, Kelihos_ver3 2942, Vundo 475, Simda 42, Tracur 751, Kelihos_ver1 398, Obfuscator.ACY 1228, Gatak 1013).
• The dataset has become a standard benchmark cited in more than 50 research papers since its release in 2015.
• Publications using the dataset cover a range of techniques including feature engineering, feature fusion, scalability, robustness, classification techniques, deep learning, and malware authorship attribution.
• The work enumerates and clusters cited papers by contribution to highlight future research directions and performance evaluation needs.