[Paper Review] AndroZoo++: Collecting Millions of Android Apps and Their Metadata for the Research Community

Li Li, Jun Gao+6 more|arXiv (Cornell University)|Sep 15, 2017

Advanced Malware Detection Techniques22 references69 citations

TL;DR

AndroZoo++ builds a growing, up-to-date repository of millions of Android apps with extensive metadata to support scalable, reproducible Android research.

ABSTRACT

We present a growing collection of Android apps collected from several sources, including the official Google Play app market and a growing collection of various metadata of those collected apps aiming at facilitating the Android-relevant research works. Our dataset by far has collected over five million apps and over 20 types of metadata such as VirusTotal reports. Our objective of collecting this dataset is to contribute to ongoing research efforts, as well as to enable new potential research topics on Android Apps. By releasing our app and metadata set to the research community, we also aim at encouraging our fellow researchers to engage in reproducible experiments. This article will be continuously updated based on the growing apps and metadata collected in the AndroZoo project. If you have specific metadata that you want to collect from AndroZoo and which are not yet provided by far, please let us know. We will thereby prioritise it in our collecting process so as to provide it to our fellow researchers in a short manner.

Motivation & Objective

• Provide a large, up-to-date collection of Android apps from multiple markets for research use.
• Offer extensive metadata to help researchers pre-select relevant apps and reduce download costs.
• Foster reproducible experiments by sharing apps and metadata with the research community.
• Support diverse research topics including security, evolution, and malware analyses.
• Encourage community-driven enhancement by gathering feedback on additional metadata needs.

Proposed method

• Develop dedicated crawlers to collect Android apps from multiple markets while minimizing load on market infrastructure.
• Implement market-specific deduplication (local within each market) and compute SHA256 checksums before archiving APKs.
• Attach rich metadata to each APK, including hashes, sizes, market source, manifest-derived fields, and security reports.
• Compute and release metadata artifacts from manifest, DEX, and releasing information to enable pre-selection and analysis without full downloads.
• Collect security-related reports (VirusTotal, AndroBugs) and other advanced metadata (piggybacking pairs, libraries, app lineages) to aid security and evolution studies.
• Provide mechanisms for researchers to request additional metadata so future collection priorities can be guided by user needs.

Experimental results

Research questions

• RQ1 How can a large-scale, up-to-date Android app collection be built from diverse markets while minimizing impact on market infrastructure?
• RQ2 What metadata beyond the APK file best supports pre-selection, filtering, and reproducible analyses of Android apps?
• RQ3 How can the collected data enable code-analysis, evolution, and malware-focused research at scale?
• RQ4 What insights can be gained about app lineages, piggybacking, and common libraries using such a dataset?

Key findings

• The dataset has grown to over five million apps with more than 20 types of metadata collected so far.
• Metadata catalog includes 24 items across APK, manifest, DEX, releasing information, security reports, and others.
• Security analyses are facilitated by VirusTotal reports and AndroBugs reports produced for the included apps.
• The dataset enables exploration of app lineages and family variants, with thousands of families and multi-version histories.
• Piggybacking pairs, common/ad libraries, and app lineage data provide insights for malware, repackaging, and evolution studies.
• The AndroZoo project emphasizes reproducibility and community engagement by providing data and soliciting metadata requests from researchers.