[Paper Review] Identifying and Characterizing Silently-Evolved Methods in the Android API

Pei Liu, Li Li+3 more|arXiv (Cornell University)|May 31, 2022

Software Engineering Research4 citations

TL;DR

This paper proposes AndroSea, a tool to identify and characterize silently-evolved methods (SEMs) in the Android API—methods whose behavior changed across versions without corresponding documentation updates. The study identifies 4,769 SEMs across ten Android API versions, finds that 363 of 562 manually analyzed SEMs involve semantic changes, and shows 957 out of 1,000 real-world apps use at least one SEM, highlighting serious compatibility risks due to undetected behavioral changes.

ABSTRACT

With over 500,000 commits and more than 700 contributors, the Android platform is undoubtedly one of the largest industrial-scale software projects. This project provides the Android API, and developers heavily rely on this API to develop their Android apps. Unfortunately, because the Android platform and its API evolve at an extremely rapid pace, app developers need to continually monitor API changes to avoid compatibility issues in their apps (\ie issues that prevent apps from working as expected when running on newer versions of the API). Despite a large number of studies on compatibility issues in the Android API, the research community has not yet investigated issues related to silently-evolved methods (SEMs). These methods are functions whose behavior might have changed but the corresponding documentation did not change accordingly. Because app developers rely on the provided documentation to evolve their apps, changes to methods that are not suitably documented may lead to unexpected failures in the apps using these methods. To shed light on this type of issue, we conducted a large-scale empirical study in which we identified and characterized SEMs across ten versions of the Android API. In the study, we identified SEMs, characterized the nature of the changes, and analyzed the impact of SEMs on a set of 1,000 real-world Android apps. Our experimental results show that SEMs do exist in the Android API, and that 957 of the apps we considered use at least one SEM. Based on these results, we argue that the Android platform developers should take actions to avoid introducing SEMs, especially those involving semantic changes. This situation highlights the need for automated techniques and tools to help Android practitioners in this task.

Motivation & Objective

• To investigate whether and how behavioral changes in Android API methods occur without corresponding documentation updates.
• To identify and characterize silently-evolved methods (SEMs) across multiple Android API versions.
• To assess the real-world impact of SEMs on existing Android applications.
• To highlight the risk of compatibility failures due to undocumented semantic changes in widely used APIs.
• To advocate for improved documentation practices and automated detection tools in Android framework development.

Proposed method

• Built AndroSea, a prototype tool that analyzes Android framework source code to detect SEMs by comparing method implementations across consecutive API versions.
• Automatically identified SEMs by detecting behavioral differences in method implementations while preserving identical Javadoc comments.
• Conducted a manual analysis of a statistically significant sample of 562 SEMs to classify the nature of changes (e.g., syntactic vs. semantic).
• Collected and analyzed usage of SEMs in a corpus of 1,000 real-world Android apps to assess real-world impact.
• Used version checks in app code as a signal to infer whether developers are mitigating SEM-related compatibility issues.
• Applied empirical analysis across ten Android API releases (from 1.5 to 10.0) to ensure broad coverage of evolution patterns.

Figure 1: High-level overview of the AndroSea workflow.

Experimental results

Research questions

• RQ1 To what extent do silently-evolved methods (SEMs) exist in the Android API across multiple versions?
• RQ2 What types of changes (syntactic or semantic) are most common in SEMs, and how do they affect API compatibility?
• RQ3 How frequently are SEMs used in real-world Android applications, and do developers detect or mitigate these issues?
• RQ4 Are there patterns in how SEMs are introduced, such as being added only once or in specific API categories?
• RQ5 To what extent do existing app practices, such as API version checks, indicate awareness of SEM-induced compatibility risks?

Key findings

• A total of 4,769 SEMs were identified across ten Android API versions, with 2,271 being publicly accessible methods.
• Of the 562 SEMs manually analyzed, 363 (64.6%) involved semantic changes, indicating a high risk of runtime failures.
• 957 out of 1,000 real-world Android apps in the study were found to use at least one publicly accessible SEM.
• The majority of SEMs (especially PASEMs) are introduced only once, suggesting that framework maintainers may not be fully aware of such changes.
• Many apps that use SEMs do not employ API version checks, indicating a lack of mitigation despite potential compatibility risks.
• The presence of version checks in some apps confirms that developers are aware of API evolution issues, but not all are proactively addressing SEM-specific risks.

(a) SEMs