[Paper Review] HiTyper: A Hybrid Static Type Inference Framework with Neural Prediction.

Yün Peng, Zongjie Li+4 more|arXiv (Cornell University)|May 8, 2021

Software Engineering Research30 references4 citations

TL;DR

HiTyper is a hybrid static and deep learning-based type inference framework that improves type prediction accuracy in dynamic languages by integrating static type inference with neural models. It constructs a type graph for type flow analysis, uses static constraints to infer types, and applies a type correction algorithm to refine predictions from a SOTA deep learning model, achieving a 12.7% improvement in top-1 F1-score and filtering 50.6% of incorrect candidate types.

ABSTRACT

Type inference for dynamic programming languages is an important yet challenging task. By leveraging the natural language information of existing human annotations, deep neural networks outperform other traditional techniques and become the state-of-the-art (SOTA) in this task. However, they are facing some new challenges, such as fixed type set, type drift, type correctness, and composite type prediction. To mitigate the challenges, in this paper, we propose a hybrid type inference framework named HiTyper, which integrates static inference into deep learning (DL) models for more accurate type prediction. Specifically, HiTyper creates a new syntax graph for each program, called type graph, illustrating the type flow among all variables in the program. Based on the type graph, HiTyper statically infers the types of the variables with appropriate static constraints. HiTyper then adopts a SOTA DL model to predict the types of other variables that cannot be inferred statically, during which process a type correction algorithm is employed to validate and correct the types recommended by the DL model. Extensive experiments show that HiTyper outperforms the SOTA DL approach by 12.7% in terms of top-1 F1-score. Moreover, HiTyper filters out 50.6% of incorrect candidate types recommended by the SOTA DL model, indicating that HiTyper could improve the correctness of predicted types. Case studies also demonstrate the capability of HiTyper in alleviating the fixed type set issue, and in handling type drift and complicated types such as composite data types.

Motivation & Objective

• Address the limitations of state-of-the-art deep learning models in type inference, including fixed type sets, type drift, and composite type prediction.
• Improve type prediction correctness and robustness by integrating static type inference with neural models.
• Mitigate issues such as type drift and incorrect candidate types in dynamic language type inference.
• Enable accurate prediction of complex composite types that are challenging for pure neural models.

Proposed method

• Constructs a type graph for each program to model type flow among variables, capturing dependencies and data flow.
• Applies static inference with appropriate constraints to determine types for variables that can be deduced without learning.
• Employs a SOTA deep learning model to predict types for variables not inferable via static analysis.
• Introduces a type correction algorithm to validate and refine the neural model's predictions using static constraints and type consistency checks.
• Combines static and learned predictions in a hybrid framework to improve overall accuracy and correctness.
• Uses the type graph as a structural backbone to guide both static inference and neural prediction, ensuring consistency.

Experimental results

Research questions

• RQ1 Can integrating static type inference with deep learning improve the accuracy of type prediction in dynamic languages?
• RQ2 To what extent can the hybrid approach reduce incorrect candidate types recommended by deep learning models?
• RQ3 How effectively does the framework handle composite data types and complex type structures?
• RQ4 Does the framework alleviate the fixed type set problem common in neural type inference models?
• RQ5 Can the type correction algorithm significantly improve the correctness of predicted types compared to pure deep learning approaches?

Key findings

• HiTyper improves the top-1 F1-score by 12.7% compared to the state-of-the-art deep learning model.
• The framework filters out 50.6% of incorrect candidate types recommended by the SOTA deep learning model, significantly enhancing prediction correctness.
• HiTyper effectively mitigates the fixed type set issue by allowing dynamic expansion of valid types through static constraints.
• The system demonstrates improved handling of type drift by validating predictions against static consistency rules.
• Case studies confirm HiTyper's capability in accurately predicting complex composite types that are difficult for pure neural models.
• The integration of static inference with deep learning leads to more reliable and consistent type predictions across diverse program structures.