[Paper Review] On The Effect Of Code Review On Code Smells

Luca Pascarella, Davide Spadini+2 more|arXiv (Cornell University)|Dec 20, 2019

Software Engineering Research81 references4 citations

TL;DR

This study investigates how code reviews affect code smell severity using 21,879 code reviews from seven Java open-source projects. It finds that active and participatory code reviews significantly reduce code smell severity, but this reduction is typically a side effect of unrelated code changes rather than intentional design improvements.

ABSTRACT

Code smells are symptoms of poor design quality. Since code review is a process that also aims at improving code quality, we investigate whether and how code review influences the severity of code smells. In this study, we analyze more than 21,000 code reviews belonging to seven Java open-source projects; we find that active and participated code reviews have a significant influence on the likelihood of reducing the severity of code smells. This result seems to confirm the expectations around code review's influence on code quality. However, by manually investigating 365 cases in which the severity of a code smell in a file was reduced with a review, we found that-in 95% of the cases-the reduction was a side effect of changes that reviewers requested on matters unrelated to code smells. Data and materials [https://doi.org/10.5281/zenodo.3588501].

Motivation & Objective

• To investigate whether and how code reviews influence the severity of code smells in software systems.
• To determine if code review activity (e.g., participation, comments) correlates with reductions in code smell severity.
• To understand whether specific code smells are more likely to be reduced during code reviews compared to others.
• To analyze the root causes behind code smell severity reductions to assess whether they result from intentional design discussions or side effects.
• To inform future tooling and practices by identifying gaps in developer awareness and code review tool support for code smells.

Proposed method

• Collected and analyzed 21,879 code reviews from seven open-source Java projects using version control and issue tracking data.
• Measured code smell severity before and after each code review at the file level using six well-known code smells.
• Applied statistical analysis to assess the relationship between code review activity (e.g., number of reviewers, comment volume) and code smell severity changes.
• Conducted a manual inspection of 365 cases where code smell severity decreased to classify the root causes of the reduction.
• Used a fine-grained, file-level comparison to isolate the effect of code reviews on code smell severity, avoiding aggregate-level confounding factors.
• Developed a classification of causes behind severity reductions to distinguish between intentional design improvements and incidental changes.

Experimental results

Research questions

• RQ1 To what extent does active and participated code review reduce the severity of code smells in reviewed files?
• RQ2 Which specific code smells are most likely to see a reduction in severity during code reviews?
• RQ3 Are reductions in code smell severity primarily due to intentional discussions about design quality or incidental side effects of unrelated changes?
• RQ4 How do the characteristics of code reviews (e.g., number of reviewers, comment volume) correlate with code smell severity changes?
• RQ5 What are the main causes behind the observed reductions in code smell severity during code reviews?

Key findings

• Active and participated code reviews significantly increase the likelihood of reducing code smell severity in reviewed files.
• In 95% of cases where code smell severity decreased, the reduction was a side effect of changes requested for reasons unrelated to code smells.
• The study found no strong correlation between the perceived criticality of code smells (as reported in prior studies) and their likelihood of being reduced during code reviews.
• Code smell severity reductions were not driven by direct discussions on design quality, indicating low developer awareness or prioritization of such issues during reviews.
• The results suggest that current code review tools do not adequately highlight code smells, leading to missed opportunities for intentional refactoring.
• Only a small fraction of code smell improvements were the result of deliberate design discussions, underscoring the need for better tooling and visualization in code review workflows.