[Paper Review] Generative Code Modeling with Graphs

Marc Brockschmidt, Miltiadis Allamanis+2 more|arXiv (Cornell University)|May 22, 2018

Software Engineering ResearchComputer Science27 references76 citations

TL;DR

Presents a graph-based generative model for code that interleaves grammar-driven expansion with graph augmentation and neural message passing to generate semantically meaningful expressions, outperforming strong baselines.

ABSTRACT

Generative models for source code are an interesting structured prediction problem, requiring to reason about both hard syntactic and semantic constraints as well as about natural, likely programs. We present a novel model for this problem that uses a graph to represent the intermediate state of the generated output. The generative procedure interleaves grammar-driven expansion steps with graph augmentation and neural message passing steps. An experimental evaluation shows that our new model can generate semantically meaningful expressions, outperforming a range of strong baselines.

Motivation & Objective

• Motivate generative code models that respect both syntactic structure and semantic constraints.
• Propose a graph-based decoder that augments ASTs with attribute-like relations and uses neural message passing.
• Introduce ExprGen, a CodeGen task focused on generating semantically complex expressions conditioned on code context.
• Evaluate the proposed graph-based approach against strong baselines on a large ExprGen dataset.

Proposed method

• Encode code context with existing encoders to obtain context and per-variable representations.
• Construct an augmented syntax graph by adding attribute edges (inherited/synthesized) and relation edges.
• Represent attributes as neural nodes and compute h_v via a Graph Neural Network with edge-type specific transforms.
• Decode by selecting production rules, variables, and literals using learned classifiers and pointer mechanisms.
• Train end-to-end with maximum likelihood, optionally augmenting edge messages with labeled edge types and attention.
• Optionally extend message passing with attention and labeled edges to improve information flow.

Experimental results

Research questions

• RQ1 Can a graph-augmented, grammar-driven decoder generate semantically valid code expressions conditioned on context?
• RQ2 Does integrating attribute grammars and neural message passing improve semantic correctness (well-typedness) and exact matches to ground truth compared to baselines?
• RQ3 How does the graph-based approach compare to tree-based and sequence-based decoders on the ExprGen task, especially under cross-project transfer?
• RQ4 What impact do labeled edges and enhanced production conditioning have on generation quality?

Key findings

• The graph-augmented model (NAG) generally achieves better semantic metrics and robustness to transfer than baselines.
• On seen-project test data, NAG achieves higher well-typed and Acc@5 scores than other graph variants.
• On unseen-project test data, NAG remains competitive, often outperforming non-graph baselines in semantic accuracy.
• PHOG baselines conditioned only on left-side context perform worse on semantic metrics than graph-based decoders.
• Overall, NAG shows strongest performance across perplexity, well-typedness, and retrieval of ground-truth expressions.
• The results indicate richer information flow via attribute graphs enhances generation quality and transferability.