[论文解读] Dynamic Neural Program Embedding for Program Repair

Ke Wang, Rishabh Singh+1 more|arXiv (Cornell University)|Nov 20, 2017

Software Engineering Research参考文献 12被引用 54

一句话总结

本文提出从运行时执行轨迹学习的语义动态程序嵌入，用于改进程序分析任务，包括错误分类和修复效率，优于基于语法的嵌入。

ABSTRACT

Neural program embeddings have shown much promise recently for a variety of program analysis tasks, including program synthesis, program repair, fault localization, etc. However, most existing program embeddings are based on syntactic features of programs, such as raw token sequences or abstract syntax trees. Unlike images and text, a program has an unambiguous semantic meaning that can be difficult to capture by only considering its syntax (i.e. syntactically similar pro- grams can exhibit vastly different run-time behavior), which makes syntax-based program embeddings fundamentally limited. This paper proposes a novel semantic program embedding that is learned from program execution traces. Our key insight is that program states expressed as sequential tuples of live variable values not only captures program semantics more precisely, but also offer a more natural fit for Recurrent Neural Networks to model. We evaluate different syntactic and semantic program embeddings on predicting the types of errors that students make in their submissions to an introductory programming class and two exercises on the CodeHunt education platform. Evaluation results show that our new semantic program embedding significantly outperforms the syntactic program embeddings based on token sequences and abstract syntax trees. In addition, we augment a search-based program repair system with the predictions obtained from our se- mantic embedding, and show that search efficiency is also significantly improved.

研究动机与目标

• 展示基于语法的程序表示在捕捉语义方面的基本局限性。
• 引入从程序执行轨迹（变量轨迹、状态轨迹，以及强依赖约束的嵌入）学习的动态程序嵌入，以捕捉语义。
• 在预测编程作业中的常见学生错误模式方面评估嵌入。
• 证明在语义预测引导下，动态嵌入能提升基于搜索的程序修复效率。

提出的方法

• 将程序执行表示为动态轨迹：变量轨迹、状态轨迹，以及依赖强化的混合形式。
• 使用基于 GRU 的 RNN 对轨迹进行编码，并通过池化进行组合以产生程序嵌入。
• 在变量轨迹之间强制数据/控制依赖，以更好地捕捉程序语义。
• 训练并评估嵌入以预测错误模式，并将其集成到 SarfGen 以实现引导修复。
• 将动态嵌入与语法基线进行比较：运行时轨迹、令牌和 AST 编码器。

实验结果

研究问题

• RQ1 从程序执行轨迹学习的动态嵌入是否能在错误模式预测方面优于基于语法的嵌入？
• RQ2 在用于优先修正时，动态嵌入是否能提高基于搜索的程序修复系统的效率？
• RQ3 哪种嵌入策略（变量轨迹、状态轨迹，或依赖强化）在修复任务中最好地捕捉程序语义？
• RQ4 动态嵌入在不同编程问题和数据集上的泛化能力如何？

主要发现

编程问题	变量轨迹	状态轨迹	依赖强化	运行时句法轨迹	令牌	AST
Print Chessboard	93.9%	95.3%	99.3%	26.3%	16.8%	16.2%
Count Parentheses	92.7%	93.8%	98.8%	25.5%	19.3%	21.7%
Generate Binary Digits	92.1%	94.5%	99.2%	23.8%	21.2%	20.9%

• 动态嵌入在预测常见学生错误模式方面显著优于基于语法的嵌入（准确率>92% 对 <27%）。
• 通过在编码过程中整合变量依赖，依赖强化的嵌入提供强语义感知表示。
• 用动态嵌入引导 SarfGen 在修复时间上带来显著加速，尤其是修复数量增加时。
• 基于语法的轨迹（运行时）相对于动态轨迹表现较差，凸显了语义差距。