[Paper Review] Android Malware Clustering using Community Detection on Android Packages Similarity Network

ElMouatez Billah Karbab, Mourad Debbabi+2 more|arXiv (Cornell University)|May 12, 2020

Advanced Malware Detection Techniques82 references4 citations

TL;DR

This paper proposes Cypider, a scalable, unsupervised framework for Android malware clustering and fingerprinting that constructs a similarity network from APK features and applies community detection to identify malicious communities. It achieves 60–80% coverage and 85–99% precision in detecting malicious communities, with community fingerprints showing 86–94% accuracy in identifying malware families and benign apps.

ABSTRACT

The daily amount of Android malicious applications (apps) targeting the app repositories is increasing, and their number is overwhelming the process of fingerprinting. To address this issue, we propose an enhanced Cypider framework, a set of techniques and tools aiming to perform a systematic detection of mobile malware by building a scalable and obfuscation resilient similarity network infrastructure of malicious apps. Our approach is based on our proposed concept, namely malicious community, in which we consider malicious instances that share common features are the most likely part of the same malware family. Using this concept, we presumably assume that multiple similar Android apps with different authors are most likely to be malicious. Specifically, Cypider leverages this assumption for the detection of variants of known malware families and zero-day malicious apps. Cypider applies community detection algorithms on the similarity network, which extracts sub-graphs considered as suspicious and possibly malicious communities. Furthermore, we propose a novel fingerprinting technique, namely community fingerprint, based on a one-class machine learning model for each malicious community. Besides, we proposed an enhanced Cypider framework, which requires less memory, x650, and less time to build the similarity network, x700, compared to the original version, without affecting the fingerprinting performance of the framework. We introduce a systematic approach to locate the best threshold on different feature content vectors, which simplifies the overall detection process.

Motivation & Objective

• Address the growing challenge of scaling Android malware analysis due to the daily influx of 10,000+ new malicious apps.
• Reduce reliance on manual analysis and signature-based detection, which fail against obfuscated or zero-day malware.
• Enable automated detection of both known malware variants and previously unseen (zero-day) malware families.
• Develop a lightweight, efficient framework that scales to large datasets without sacrificing detection accuracy.
• Introduce a novel community fingerprinting technique based on one-class machine learning to improve malware family identification.

Proposed method

• Construct a similarity network by representing each Android app as a feature vector and connecting apps with high similarity on any feature.
• Apply community detection algorithms to extract densely connected subgraphs, interpreted as potential malicious communities.
• Introduce the concept of a 'malicious community'—apps sharing common features, especially code or components, are likely part of the same malware family.
• Develop a community fingerprint using one-class machine learning models trained per detected community for efficient malware family identification.
• Optimize the framework for memory and time efficiency, reducing similarity network construction time by ~700× and memory usage by ~650% compared to the original version.
• Systematically determine optimal similarity thresholds across different feature vectors to simplify and improve detection accuracy.

Experimental results

Research questions

• RQ1 Can community detection on a similarity network of Android APKs effectively identify clusters of malicious apps, including variants and zero-day malware?
• RQ2 How does the proposed Cypider framework compare in efficiency and accuracy to existing similarity-based malware detection methods?
• RQ3 To what extent can community fingerprints based on one-class learning models detect known malware families and distinguish them from benign apps?
• RQ4 What is the optimal threshold for similarity thresholds across different feature vectors to maximize detection coverage and precision?
• RQ5 Can the framework scale to large datasets (e.g., 110k apps) while maintaining high performance and low resource consumption?

Key findings

• Cypider achieves 60–80% coverage of the malware dataset in a single detection iteration, with 85–99% precision in identified malicious communities.
• The framework reduces similarity network construction time by approximately 700× and memory usage by about 650% compared to the original version, without degrading fingerprinting performance.
• Community fingerprints achieve 86% detection accuracy for malware families, 93% for general malware, and 94% for benign apps.
• The systematic threshold selection process simplifies detection and enhances consistency across different feature sets.
• The framework successfully identifies both repackaged malware and zero-day threats by detecting high-similarity clusters across diverse APK features.
• Evaluation on a dataset of 110,000 apps shows that community detection completes in approximately 60 seconds, demonstrating high scalability.