[Paper Review] Inferring Javascript types using Graph Neural Networks

Jessica Schrouff, Kai Wohlfahrt+2 more|arXiv (Cornell University)|May 16, 2019

Software Engineering Research15 references4 citations

TL;DR

This paper proposes a graph neural network (GNN) model that infers JavaScript token types by representing source code as a graph with syntactic and semantic edges. The model achieves over 90% accuracy in predicting eight JavaScript types, outperforming prior work and demonstrating strong potential for integration into code editors or static analysis tools.

ABSTRACT

The recent use of `Big Code' with state-of-the-art deep learning methods offers promising avenues to ease program source code writing and correction. As a first step towards automatic code repair, we implemented a graph neural network model that predicts token types for Javascript programs. The predictions achieve an accuracy above $90\%$, which improves on previous similar work.

Motivation & Objective

• To address the challenge of type inference in dynamically-typed JavaScript, where type errors can go undetected during execution.
• To develop a deep learning model that predicts token-level types without relying on static type annotations.
• To improve upon existing type inference methods by leveraging graph-based neural networks on abstract syntax trees (ASTs).
• To create a deployable system suitable for integration into real-time development environments like online code editors or code review tools.
• To evaluate the effectiveness of GNNs in modeling program structure for type prediction in JavaScript.

Proposed method

• Each JavaScript source file is parsed into an abstract syntax tree (AST), with nodes representing AST tokens and edges encoding syntactic (e.g., 'child') and semantic (e.g., 'defined by') relationships.
• Node features include AST node type (144 classes), properties (107 types), and string values (capped at 16 characters), each processed through dedicated embedding layers.
• Node embeddings are combined and passed through feed-forward neural networks with batch normalization, dropout, and ReLU activation.
• Two GNN architectures are evaluated: Graph Convolutional Network (GCN) and Gated Graph Neural Network (GGNN), both using message passing to aggregate neighborhood information.
• The GCN uses separate linear layers per edge type, sums messages, and applies ReLU; the GGNN uses GRU cells for state updates with tied weights across layers.
• A master node is explored in GGNN to model long-range dependencies, though it shows no performance benefit in the final configuration.

Experimental results

Research questions

• RQ1 Can graph neural networks effectively infer JavaScript token types from AST-based program representations without relying on static type information?
• RQ2 How do different GNN architectures (GCN vs. GGNN) compare in performance for JavaScript type inference on real-world code?
• RQ3 What hyper-parameter configurations yield optimal type prediction accuracy for GNNs in this context?
• RQ4 Does the inclusion of a master node improve type prediction performance in GNNs trained on AST graphs?
• RQ5 To what extent can this model generalize to unseen repositories and achieve high accuracy in real-world development settings?

Key findings

• The proposed GNN model achieves a type prediction accuracy exceeding 90%, significantly outperforming previous methods in the same task.
• The GCN model performs best with no encoding feed-forward blocks, a hidden size of 64, a dropout rate of 0.1, 7 to 10 message passing layers, and 1 decoding layer.
• The GGNN model performs best with 1 encoding layer, a hidden size of 128, no dropout, 5 message passing layers, no master node, and 1 decoding layer.
• The inclusion of a master node did not improve performance and was excluded from the final GGNN configuration, suggesting that long-range connections are not beneficial on AST graphs.
• The hyper-parameter search revealed that higher numbers of message passing layers and decoding layers generally improved performance, especially in GCN.
• The model generalizes well to unseen repositories, as training and testing were split at the repository level to avoid data leakage.