[Paper Review] Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN

Weiwei Hu, Ying Tan|arXiv (Cornell University)|Feb 20, 2017

Advanced Malware Detection Techniques17 references366 citations

TL;DR

MalGAN uses a generator and a substitute detector to craft adversarial malware that can fool black-box malware detectors, achieving near-zero true positive rates across multiple classifiers, and exposing weaknesses in retraining defenses.

ABSTRACT

Machine learning has been used to detect new malware in recent years, while malware authors have strong motivation to attack such algorithms. Malware authors usually have no access to the detailed structures and parameters of the machine learning models used by malware detection systems, and therefore they can only perform black-box attacks. This paper proposes a generative adversarial network (GAN) based algorithm named MalGAN to generate adversarial malware examples, which are able to bypass black-box machine learning based detection models. MalGAN uses a substitute detector to fit the black-box malware detection system. A generative network is trained to minimize the generated adversarial examples' malicious probabilities predicted by the substitute detector. The superiority of MalGAN over traditional gradient based adversarial example generation algorithms is that MalGAN is able to decrease the detection rate to nearly zero and make the retraining based defensive method against adversarial examples hard to work.

Motivation & Objective

• Motivate robustness challenges of machine learning–based malware detectors against adversarial attacks.
• Propose a GAN-based framework (MalGAN) to generate adversarial malware without access to the detector’s internals.
• Enable malware authors to bypass black-box detectors by learning a substitute model and using feedback from the detector.
• Demonstrate the approach’s effectiveness across multiple detector types and compare with gradient-based white/gray-box methods.
• Explore the impact of retraining defenses and the dynamics of adversarial adaptation.

Proposed method

• MalGAN comprises a generator G and a substitute detector D, both neural networks, operating with binary API features from PE programs.
• The generator takes a malware feature vector m and a noise vector z, producing o; a binarization step converts o to o' and the final adversarial sample m' = m OR o'.
• A smooth function G ensures gradient flow through non-modified bits, enabling backpropagation to train the generator despite binary outputs.
• The substitute detector D is trained to mimic the black-box detector by learning from detector feedback on adversarial malware and benign samples, without using ground-truth labels from the black-box.
• Generator loss L_G encourages misclassification of adversarial samples by the substitute detector, while the substitute detector loss L_D aligns with the black-box detector’s outputs.
• Training alternates updating D (minimizing L_D) and G (minimizing L_G) using mini-batches from malware and benign datasets.

Experimental results

Research questions

• RQ1 Can MalGAN generate adversarial malware that reliably bypasses a black-box detector without knowledge of its internals?
• RQ2 How well can a substitute detector approximate the black-box detector across different classifier architectures?
• RQ3 What is the transferability of adversarial malware across various detector models (RF, LR, DT, SVM, MLP, VOTE)?
• RQ4 How does retraining the black-box detector affect the effectiveness of MalGAN?
• RQ5 How does MalGAN compare to gradient-based white/gray-box methods for generating adversarial malware?

Key findings

• MalGAN reduces the true positive rate of various black-box detectors from high original levels to near zero on adversarial examples when trained on the same data.
• When trained with different data, MalGAN still drives TPR on adversarial examples to near zero for most classifiers, though some gaps remain (e.g., LR/MLP/SVM reaching near-zero, RF/DT showing small nonzero TPR).
• MalGAN's adversarial examples transfer across diverse detector architectures and are more effective than gradient-based binary attacks under black-box assumptions.
• Defensive retraining of the black-box detector can drive adversarial samples’ detection back to 100% (zero adversarial misses) in a single retraining epoch, but the detector’s effectiveness collapses again if MalGAN is retrained against the updated detector.
• Compared to gradient-based methods, MalGAN achieves far stronger evasion under a black-box setting, avoiding the generalization issues caused by shifting distributions of adversarial examples.