[Paper Review] Semi-supervised classification for dynamic Android malware detection

Li Chen, Mingwei Zhang+2 more|arXiv (Cornell University)|Apr 19, 2017

Advanced Malware Detection Techniques22 references14 citations

TL;DR

This paper proposes a model-based semi-supervised (MBSS) classification framework for dynamic Android malware detection using API call logs, leveraging both labeled and unlabeled APKs to estimate a Gaussian mixture model via conditional expectation-maximization. MBSS achieves 98% accuracy in-sample and maintains 90% detection rate out-of-sample under strong similarity, significantly outperforming SVM, kNN, and LDA when test data deviate from training distribution.

ABSTRACT

A growing number of threats to Android phones creates challenges for malware detection. Manually labeling the samples into benign or different malicious families requires tremendous human efforts, while it is comparably easy and cheap to obtain a large amount of unlabeled APKs from various sources. Moreover, the fast-paced evolution of Android malware continuously generates derivative malware families. These families often contain new signatures, which can escape detection when using static analysis. These practical challenges can also cause traditional supervised machine learning algorithms to degrade in performance. In this paper, we propose a framework that uses model-based semi-supervised (MBSS) classification scheme on the dynamic Android API call logs. The semi-supervised approach efficiently uses the labeled and unlabeled APKs to estimate a finite mixture model of Gaussian distributions via conditional expectation-maximization and efficiently detects malwares during out-of-sample testing. We compare MBSS with the popular malware detection classifiers such as support vector machine (SVM), $k$-nearest neighbor (kNN) and linear discriminant analysis (LDA). Under the ideal classification setting, MBSS has competitive performance with 98\% accuracy and very low false positive rate for in-sample classification. For out-of-sample testing, the out-of-sample test data exhibit similar behavior of retrieving phone information and sending to the network, compared with in-sample training set. When this similarity is strong, MBSS and SVM with linear kernel maintain 90\% detection rate while $k$NN and LDA suffer great performance degradation. When this similarity is slightly weaker, all classifiers degrade in performance, but MBSS still performs significantly better than other classifiers.

Motivation & Objective

• Address the challenge of limited labeled Android malware samples and the growing volume of unlabeled APKs.
• Overcome performance degradation of supervised classifiers on out-of-sample malware with non-identical distributions.
• Develop a robust malware detection method that leverages both labeled and unlabeled dynamic behavior data.
• Improve malware triage efficiency by identifying high-probability malicious samples from large unlabeled datasets.
• Ensure resilience against adversarial manipulation in feature extraction, particularly for N-gram-based methods.

Proposed method

• Collect dynamic API call logs by executing APKs in an emulator environment.
• Model the data using a finite mixture of Gaussian distributions to represent benign and malicious behaviors.
• Apply conditional expectation-maximization (CEM) to estimate model parameters using both labeled and unlabeled data.
• Use Bayesian Information Criterion (BIC) for optimal model selection among candidate mixture models.
• Classify new samples using posterior probabilities derived from the learned mixture model.
• Compare MBSS performance against SVM (linear and radial), kNN, and LDA under varying out-of-sample similarity conditions.

Experimental results

Research questions

• RQ1 Can semi-supervised learning improve malware detection robustness when test data are not identically distributed to training data?
• RQ2 How does MBSS perform compared to traditional classifiers (SVM, kNN, LDA) on out-of-sample Android malware with evolving signatures?
• RQ3 To what extent can unlabeled APKs enhance detection accuracy and reduce false positives in dynamic malware analysis?
• RQ4 Does using uni-gram features instead of higher-order N-grams improve resilience to adversarial manipulation in malware detection?
• RQ5 How does the mixture-of-Gaussians assumption hold for real-world Android API call log distributions?

Key findings

• MBSS achieved 98% accuracy and a very low false positive rate in in-sample classification, demonstrating competitive performance against standard classifiers.
• Under strong similarity between training and out-of-sample data, MBSS and linear SVM maintained 90% detection rate, while kNN and LDA suffered significant performance degradation.
• When similarity weakened, all classifiers degraded, but MBSS still outperformed SVM, kNN, and LDA in detection rate.
• The MBSS framework effectively leverages unlabeled data to improve detection robustness, especially in scenarios with distributional shift.
• Using uni-gram features instead of higher-order N-grams enhanced resilience to adversarial attacks in feature engineering.
• The method's performance is sensitive to deviations from the Gaussian mixture assumption, suggesting caution in highly non-Gaussian data distributions.