[Paper Review] Learning Semantic Program Embeddings with Graph Interval Neural Network

Yu Wang, Fengjuan Gao+2 more|arXiv (Cornell University)|May 18, 2020

Software Engineering Research37 references4 citations

TL;DR

This paper proposes Graph Interval Neural Network (GINN), a novel GNN architecture that learns semantic program embeddings by focusing on interval structures—primarily loops—in control flow graphs. By abstracting programs into hierarchical intervals, GINN improves both precision and scalability over standard GNNs, outperforming state-of-the-art models in variable misuse prediction, method name prediction, and neural bug detection on real-world Java codebases.

ABSTRACT

Learning distributed representations of source code has been a challenging task for machine learning models. Earlier works treated programs as text so that natural language methods can be readily applied. Unfortunately, such approaches do not capitalize on the rich structural information possessed by source code. Of late, Graph Neural Network (GNN) was proposed to learn embeddings of programs from their graph representations. Due to the homogeneous and expensive message-passing procedure, GNN can suffer from precision issues, especially when dealing with programs rendered into large graphs. In this paper, we present a new graph neural architecture, called Graph Interval Neural Network (GINN), to tackle the weaknesses of the existing GNN. Unlike the standard GNN, GINN generalizes from a curated graph representation obtained through an abstraction method designed to aid models to learn. In particular, GINN focuses exclusively on intervals for mining the feature representation of a program, furthermore, GINN operates on a hierarchy of intervals for scaling the learning to large graphs. We evaluate GINN for two popular downstream applications: variable misuse prediction and method name prediction. Results show in both cases GINN outperforms the state-of-the-art models by a comfortable margin. We have also created a neural bug detector based on GINN to catch null pointer deference bugs in Java code. While learning from the same 9,000 methods extracted from 64 projects, GINN-based bug detector significantly outperforms GNN-based bug detector on 13 unseen test projects. Next, we deploy our trained GINN-based bug detector and Facebook Infer to scan the codebase of 20 highly starred projects on GitHub. Through our manual inspection, we confirm 38 bugs out of 102 warnings raised by GINN-based bug detector compared to 34 bugs out of 129 warnings for Facebook Infer.

Motivation & Objective

• To address the limitations of standard Graph Neural Networks (GNNs) in learning precise, scalable program embeddings from source code.
• To exploit program-specific structural characteristics—particularly loops—by abstracting control flow graphs into hierarchical intervals.
• To develop a general-purpose deep learning model that captures semantic program features more effectively than existing text-based or graph-based models.
• To improve downstream program analysis tasks such as bug detection, variable misuse prediction, and method name prediction.
• To demonstrate the effectiveness of interval-based abstraction in reducing computational cost while maintaining or improving model accuracy.

Proposed method

• GINN uses a novel abstraction method to transform control flow graphs into hierarchical intervals, where each interval represents a loop construct.
• The model focuses exclusively on these intervals to extract deep semantic features, reducing noise from non-essential graph components.
• GINN operates on a hierarchy of intervals, enabling efficient scaling to large programs by abstracting nested loop structures.
• It leverages the standard GNN message-passing mechanism but applies it only within and across intervals, not across the entire graph.
• The abstraction ensures that only semantically significant subgraphs (loops) are processed, improving learning efficiency and precision.
• GINN is trained end-to-end for downstream tasks such as variable misuse prediction, method name prediction, and bug detection.

Experimental results

Research questions

• RQ1 Can focusing on interval structures (e.g., loops) in control flow graphs improve the precision of program embedding models?
• RQ2 How does hierarchical abstraction of intervals enhance scalability of GNNs on large codebases?
• RQ3 Does GINN outperform standard GNNs and prior state-of-the-art models in downstream program analysis tasks?
• RQ4 Can a GINN-based neural bug detector detect real-world bugs more effectively than existing tools like Facebook Infer?
• RQ5 To what extent does interval-based abstraction reduce the need for extensive message passing across large graphs?

Key findings

• GINN outperforms state-of-the-art models in variable misuse prediction and method name prediction, achieving a comfortable margin in both tasks.
• On 13 unseen test projects, the GINN-based bug detector detected more true positives than the GNN-based detector when trained on 9,000 methods from 64 projects.
• In a real-world evaluation on 20 highly starred GitHub projects, GINN raised 102 warnings, of which 38 were confirmed real bugs, compared to 34 out of 129 for Facebook Infer.
• Of the 38 bugs caught by GINN, 11 were fixed and 12 were confirmed (fix pending), demonstrating strong practical utility.
• The hierarchical interval abstraction enables GINN to scale efficiently to large graphs without sacrificing precision, unlike standard GNNs.
• GINN’s focus on loop intervals allows it to capture deeper semantic features, leading to more accurate and generalizable program embeddings.