[论文解读] Towards Synthesizing Complex Programs from Input-Output Examples

Xinyun Chen, Chang Liu+1 more|arXiv (Cornell University)|Jun 5, 2017

Software Engineering Research参考文献 29被引用 15

一句话总结

本文提出了一种新颖的方法，通过训练神经程序来操作一种特定领域的不可微分机器，从而从输入-输出示例中合成复杂程序。该方法结合不可微分机器与两阶段强化学习算法，在输入序列长度为训练输入500倍的情况下实现了100%的测试准确率，克服了先前最先进方法在泛化能力上的失败。

ABSTRACT

In recent years, deep learning techniques have been developed to improve the performance of program synthesis from input-output examples. Albeit its significant progress, the programs that can be synthesized by state-of-the-art approaches are still simple in terms of their complexity. In this work, we move a significant step forward along this direction by proposing a new class of challenging tasks in the domain of program synthesis from input-output examples: learning a context-free parser from pairs of input programs and their parse trees. We show that this class of tasks are much more challenging than previously studied tasks, and the test accuracy of existing approaches is almost 0%. We tackle the challenges by developing three novel techniques inspired by three novel observations, which reveal the key ingredients of using deep learning to synthesize a complex program. First, the use of a non-differentiable machine is the key to effectively restrict the search space. Thus our proposed approach learns a neural program operating a domain-specific non-differentiable machine. Second, recursion is the key to achieve generalizability. Thus, we bake-in the notion of recursion in the design of our non-differentiable machine. Third, reinforcement learning is the key to learn how to operate the non-differentiable machine, but it is also hard to train the model effectively with existing reinforcement learning algorithms from a cold boot. We develop a novel two-phase reinforcement learning-based search algorithm to overcome this issue. In our evaluation, we show that using our novel approach, neural parsing programs can be learned to achieve 100% test accuracy on test inputs that are 500x longer than the training samples.

研究动机与目标

• 解决程序合成在复杂程序（超出简单字符串或数组操作）上的泛化挑战。
• 开发一种方法，能够从输入-树对中学习递归的上下文无关语言解析器，且无需执行轨迹监督。
• 克服端到端可微分模型的泛化能力差以及从冷启动开始的强化学习训练不稳定性问题。
• 证明神经程序可以完全泛化到远长于训练期间所见的输入。

提出的方法

• 设计一种支持递归且将搜索空间限制为有效程序的领域特定不可微分机器。
• 使用一种新颖的两阶段强化学习算法，训练神经程序来操作该机器，以恢复执行轨迹并训练策略。
• 将递归内置于机器的架构中，以实现对任意长输入的泛化能力。
• 将不可微分机器作为可微分接口，约束神经程序仅生成有效且结构化的程序。
• 将训练过程分解为两个阶段：第一阶段，搜索可能的执行轨迹；第二阶段，使用恢复的轨迹作为监督信号训练神经策略。
• 利用机器的递归结构，使神经程序能够泛化至训练序列长度之外。

实验结果

研究问题

• RQ1 当学习解析上下文无关语法时，神经程序是否可以被训练以泛化到远长于训练期间所见的输入？
• RQ2 在无执行轨迹监督的情况下，如何从冷启动开始稳定强化学习训练？
• RQ3 不可微分机器在将搜索空间限制为有效程序并提升泛化能力方面发挥什么作用？
• RQ4 能否在神经程序的执行机器中有效编码递归，以实现完美泛化？

主要发现

• 所提出的方法在输入序列长度为最长训练序列500倍的情况下实现了100%的测试准确率，证明了完全泛化能力。
• 现有最先进方法（包括RobustFill和StackRNN）在相同长输入上的测试准确率接近0%，凸显了泛化挑战。
• 两阶段强化学习算法成功避免了局部极小值，实现了从冷启动的有效训练，优于先前方法。
• 将递归整合到不可微分机器中，是实现对任意长输入泛化能力的关键。
• 使用不可微分机器能有效约束神经程序仅生成有效且结构化的程序，防止对训练数据的过拟合。