[论文解读] CORE: Automating Review Recommendation for Code Changes

Jingkai Siow, Cuiyun Gao+3 more|arXiv (Cornell University)|Dec 20, 2019

Software Engineering Research参考文献 49被引用 5

一句话总结

本文提出CORE模型，一种仅使用代码变更和历史代码评审记录的深度学习模型，无需完整源代码即可推荐相关代码评审。通过结合词级别和字符级别的嵌入表示，并引入注意力机制，CORE在19个Java项目上的实验中，Recall@10相比当前最先进模型DeepMem提升131.03%，平均倒数排名（MRR）提升150.69%。

ABSTRACT

Code review is a common process that is used by developers, in which a reviewer provides useful comments or points out defects in the submitted source code changes via pull request. Code review has been widely used for both industry and open-source projects due to its capacity in early defect identification, project maintenance, and code improvement. With rapid updates on project developments, code review becomes a non-trivial and labor-intensive task for reviewers. Thus, an automated code review engine can be beneficial and useful for project development in practice. Although there exist prior studies on automating the code review process by adopting static analysis tools or deep learning techniques, they often require external sources such as partial or full source code for accurate review suggestion. In this paper, we aim at automating the code review process only based on code changes and the corresponding reviews but with better performance. The hinge of accurate code review suggestion is to learn good representations for both code changes and reviews. To achieve this with limited source, we design a multi-level embedding (i.e., word embedding and character embedding) approach to represent the semantics provided by code changes and reviews. The embeddings are then well trained through a proposed attentional deep learning model, as a whole named CORE. We evaluate the effectiveness of CORE on code changes and reviews collected from 19 popular Java projects hosted on Github. Experimental results show that our model CORE can achieve significantly better performance than the state-of-the-art model (DeepMem), with an increase of 131.03% in terms of Recall@10 and 150.69% in terms of Mean Reciprocal Rank. Qualitative general word analysis among project developers also demonstrates the performance of CORE in automating code review.

研究动机与目标

• 通过仅依赖代码变更和历史评审记录，而非完整源代码，减少代码评审人员的重复性工作负担。
• 通过学习更优的代码变更和评审的语义表示，提升代码评审推荐的准确性。
• 通过结合词级别和字符级别的嵌入表示，解决代码中未登录词（OOV）的问题。
• 构建一个仅基于代码变更与评审对的模型，学习二者之间的相关性。
• 通过定量评估和资深开发者的定性反馈，验证模型的有效性。

提出的方法

• CORE采用多层级嵌入策略，结合词级别和字符级别的嵌入表示，用于代码变更和评审内容。
• 模型使用注意力神经网络计算代码变更与评审之间的相关性得分，聚焦于语义上重要的部分。
• 词嵌入从代码标记中学习，字符嵌入则捕捉子词模式，以处理罕见或未知标记。
• 注意力机制在预测过程中动态加权代码变更和评审中的关键标记。
• 模型在GitHub代码仓库中的代码变更-评审对上进行端到端训练，无需外部代码上下文。
• 分层注意力机制使模型能够同时关注相关的代码段和评审评论。

实验结果

研究问题

• RQ1 是否可以仅使用代码变更和历史评审记录，无需完整源代码，训练一个深度学习模型来推荐相关代码评审？
• RQ2 多层级嵌入（词+字符）方法在捕捉代码变更和评审语义含义方面是否有效？
• RQ3 与基线模型相比，注意力机制是否能提升代码变更与评审之间相关性预测的性能？
• RQ4 在真实世界代码评审数据上，CORE与当前最先进模型（如DeepMem）相比，检索性能如何？
• RQ5 资深开发人员在多大程度上认为CORE推荐的评审内容具有实用性和相关性？

主要发现

• 在19个流行的Java项目上，CORE相比当前最先进模型DeepMem，Recall@10提升131.03%。
• 与DeepMem相比，CORE在平均倒数排名（MRR）上提升150.69%，表明其对相关评审的排序能力更强。
• 资深开发者的定性分析表明，CORE生成的推荐结果在实际应用中具有相关性和实用性。
• 词级别与字符级别嵌入的结合显著增强了表示学习能力，尤其在处理代码中罕见或未登录词时效果明显。
• 注意力机制能有效突出关键代码元素和评审短语，从而提升预测准确性。
• 即使在无法访问完整源代码的情况下，模型仍表现良好，具备在实际拉取请求系统中部署的实用性。