[Paper Review] Automatic Generation of Adversarial Examples for Interpreting Malware Classifiers.

Wei Song, Xuezixiang Li+4 more|arXiv (Cornell University)|Mar 6, 2020

Advanced Malware Detection Techniques52 references21 citations

TL;DR

This paper proposes a framework to generate realistic adversarial malware examples that evade real-world antivirus systems by combining code randomization and binary manipulation. It demonstrates that 24.3%–41.9% of malware samples can bypass static detectors—sometimes with just one byte change—and that some static evasions also bypass offline dynamic analysis, revealing critical vulnerabilities in commercial AVs.

ABSTRACT

Recent advances in adversarial attacks have shown that machine learning classifiers based on static analysis are vulnerable to adversarial attacks. However, real-world antivirus systems do not rely only on static classifiers, thus many of these static evasions get detected by dynamic analysis whenever the malware runs. The real question is to what extent these adversarial attacks are actually harmful to the real users? In this paper, we propose a systematic framework to create and evaluate realistic adversarial malware to evade real-world systems. We propose new adversarial attacks against real-world antivirus systems based on code randomization and binary manipulation, and use our framework to perform the attacks on 1000 malware samples and test four commercial antivirus software and two open-source classifiers. We demonstrate that the static detectors of real-world antivirus can be evaded 24.3%-41.9% of the cases and often by changing only one byte. We also find that the adversarial attacks are transferable between different antivirus up to 16% of the cases. We also tested the efficacy of the complete (i.e. static + dynamic) classifiers in protecting users. While most of the commercial antivirus use their dynamic engines to protect the users' device when the static classifiers are evaded, we are the first to demonstrate that for one commercial antivirus, static evasions can also evade the offline dynamic detectors and infect users' machines. Our framework can also help explain which features are responsible for evasion and thus can help improve the robustness of malware detectors.

Motivation & Objective

• To evaluate the real-world effectiveness of adversarial attacks on malware classifiers beyond static analysis.
• To develop a systematic framework for generating adversarial malware that evades both static and dynamic analysis in commercial antivirus systems.
• To investigate whether adversarial examples crafted for static classifiers can also bypass dynamic analysis engines.
• To identify which features in malware are most responsible for successful evasion, enabling improved detector robustness.

Proposed method

• The framework generates adversarial malware by applying code randomization and binary manipulation to 1,000 real malware samples.
• It evaluates the generated adversarial samples against four commercial antivirus engines and two open-source classifiers.
• The approach focuses on minimal code changes—such as altering a single byte—to maintain functionality while evading detection.
• Transferability of adversarial examples across different antivirus systems is measured by testing evasion success on multiple engines.
• Feature attribution techniques are used to identify which code-level features contribute most to evasion success.

Experimental results

Research questions

• RQ1 To what extent can adversarial malware generated via binary manipulation evade real-world commercial antivirus systems?
• RQ2 Can adversarial examples that evade static analysis also bypass offline dynamic analysis in commercial antivirus engines?
• RQ3 How transferable are adversarial examples across different antivirus detection systems?
• RQ4 Which code-level features are most influential in enabling evasion of malware classifiers?

Key findings

• The proposed adversarial attacks successfully evaded the static detectors of real-world antivirus systems in 24.3% to 41.9% of cases.
• In some cases, only a single-byte modification was sufficient to evade detection, highlighting the fragility of current static analysis.
• Adversarial examples showed transferability across different antivirus engines in up to 16% of cases.
• One commercial antivirus product failed to detect malware even when its static classifier was evaded, as its offline dynamic engine also failed to trigger.
• The framework enabled identification of key features responsible for evasion, offering insights for improving detector robustness.