[Paper Review] An Empirical Study on the Impact of Refactoring Activities on Evolving Client-Used APIs

Raula Gaikovina Kula, Ali Ouni+2 more|arXiv (Cornell University)|Sep 27, 2017

Software Engineering Research28 references3 citations

TL;DR

This empirical study analyzes the impact of refactoring on client-used APIs in eight popular Java libraries across 183 versions, using automated tooling to detect 12,900 refactorings and 9,700 API breakages. It finds that only 37% of client-used API breakages are directly caused by refactoring, with the remainder driven by bug fixes and feature additions, indicating that library maintainers prioritize backward compatibility despite refactoring activities.

ABSTRACT

Context: Refactoring is recognized as an effective practice to maintain evolving software systems. For software libraries, we study how library developers refactor their Application Programming Interfaces (APIs), especially when it impacts client users by breaking an API of the library. Objective: Our work aims to understand how clients that use a library API are affected by refactoring activities. We target popular libraries that potentially impact more library client users. Method: We distinguish between library APIs based on their client-usage (refereed to as client-used APIs) in order to understand the extent to which API breakages relate to refactorings. Our tool-based approach allows for a large-scale study across eight libraries (i.e., totaling 183 consecutive versions) with around 900 clients projects. Results: We find that library maintainers are less likely to break client-used API classes. Quantitatively, we find that refactoring activities break less than 37% of all client-used APIs. In a more qualitative analysis, we show two documented cases of where non-refactoring API breaking changes are motivated other maintenance issues (i.e., bug fix and new features) and involve more complex refactoring operations. Conclusion: Using our automated approach, we find that library developers are less likely to break APIs and tend to break client-used APIs when addressing these maintenance issues.

Motivation & Objective

• To understand how refactoring activities impact client-used APIs in evolving software libraries.
• To investigate the extent to which API breakages in client-used APIs are caused by refactoring versus other maintenance activities.
• To assess whether library maintainers prioritize backward compatibility when refactoring client-facing APIs.
• To provide a large-scale, tool-based dataset of API breakages and refactorings for replication and further research.

Proposed method

• Automated static analysis was used to detect refactoring operations (e.g., move_method, rename_method) in 183 consecutive versions of eight popular Java libraries.
• API breakages were identified by comparing binary compatibility between library versions using bytecode analysis.
• Client usage of APIs was determined by mining real-world client projects (around 900 projects) to identify which APIs are actively used.
• The study applied non-parametric statistical tests to analyze relationships between refactoring types and API breakage frequency.
• A replication package with the full dataset of refactorings and breakages was created and made publicly available.

Experimental results

Research questions

• RQ1 To what extent are client-used APIs broken due to refactoring activities in evolving software libraries?
• RQ2 How do the frequency and types of refactorings differ between client-used APIs and non-client-used APIs?
• RQ3 What motivates the remaining API breakages not caused by refactoring, and how do they relate to maintenance tasks like bug fixes or feature additions?
• RQ4 Are certain refactoring types (e.g., rename_method, move_field) more likely to break client-used APIs than others?

Key findings

• Only 37% of client-used API breakages are directly caused by refactoring operations, indicating that the majority of breakages stem from non-refactoring activities.
• Library maintainers are significantly less likely to break client-used APIs compared to internal or non-client-used classes.
• The remaining 63% of API breakages are primarily motivated by non-refactoring maintenance tasks such as bug fixes and feature additions.
• Simple refactorings like move_method, rename_method, and move_field are applied less frequently to client-used APIs than to other classes.
• The study reveals that complex refactorings are often involved in non-refactoring-driven breakages, suggesting deeper design or maintenance challenges.
• The publicly available dataset of 12,900 refactorings and 9,700 API breakages enables future replication and tool development in API evolution research.