[Paper Review] Learning a Static Bug Finder from Data

Yu Wang, Fengjuan Gao+2 more|arXiv (Cornell University)|Jul 12, 2019

Software Engineering Research43 references8 citations

TL;DR

This paper proposes NeurSA, a framework that trains deep learning models—specifically Graph Neural Networks with an interval-based propagation mechanism—directly from code data to detect semantic bugs like null pointer dereferences and array index errors. The approach outperforms traditional static analyzers in precision and discovers 50 new bugs in real-world projects, including 9 fixed and 3 confirmed.

ABSTRACT

We present an alternative approach to creating static bug finders. Instead of relying on human expertise, we utilize deep neural networks to train static analyzers directly from data. In particular, we frame the problem of bug finding as a classification task and train a classifier to differentiate the buggy from non-buggy programs using Graph Neural Network (GNN). Crucially, we propose a novel interval-based propagation mechanism that leads to a significantly more efficient, accurate and scalable generalization of GNN. We have realized our approach into a framework, NeurSA, and extensively evaluated it. In a cross-project prediction task, three neural bug detectors we instantiate from NeurSA are effective in catching null pointer dereference, array index out of bound and class cast bugs in unseen code. We compare NeurSA against several static analyzers (e.g. Facebook Infer and Pinpoint) on a set of null pointer dereference bugs. Results show that NeurSA is more precise in catching the real bugs and suppressing the spurious warnings. We also apply NeurSA to several popular Java projects on GitHub and discover 50 new bugs, among which 9 have been fixed, and 3 have been confirmed.

Motivation & Objective

• To address the high false positive and false negative rates in traditional static bug finders by replacing human-authored rules with data-driven learning.
• To improve the capability of Graph Neural Networks (GNNs) in detecting complex semantic bugs beyond syntactic patterns.
• To develop a scalable, general-purpose framework for creating neural static analyzers without requiring manual specification of bug patterns.
• To reduce the dependency on expert knowledge in designing static analysis rules by automating the learning process from code corpora.
• To enable effective cross-project detection of bugs such as null pointer dereferences, array index out of bounds, and class cast exceptions.

Proposed method

• The framework collects training data from multiple open-source projects, automatically labeling programs as buggy or clean based on known bug reports.
• It represents programs as code property graphs and applies a novel interval-based propagation mechanism (IBPM) to enhance GNNs' ability to learn deep semantic features.
• IBPM enables hierarchical message passing over graph intervals, improving both model expressiveness and scalability for large programs.
• The model is trained as a binary classification task to distinguish between buggy and non-buggy code snippets using node and graph-level representations.
• NeurSA performs inter-procedural analysis by propagating information across function calls and control-flow structures in the program graph.
• The framework is extensible: new bug detectors are created by retraining on labeled data for specific bug types, without changing the core architecture.

Experimental results

Research questions

• RQ1 Can a deep learning model trained directly from code data detect complex semantic bugs more effectively than rule-based static analyzers?
• RQ2 How does the proposed interval-based propagation mechanism improve the performance and scalability of GNNs in program analysis?
• RQ3 To what extent can a data-driven neural approach generalize across projects and detect previously unseen bugs?
• RQ4 Can the framework discover real-world bugs in large, actively maintained open-source projects?
• RQ5 How does the model's performance compare to state-of-the-art static analyzers like Facebook Infer and Pinpoint in terms of precision and recall?

Key findings

• NeurSA outperforms Facebook Infer and Pinpoint in detecting null pointer dereference bugs, showing higher precision and fewer false positives.
• The framework discovered 50 new bugs in popular GitHub projects, with 9 confirmed as fixed and 3 confirmed by project maintainers.
• Three neural bug detectors instantiated from NeurSA successfully detected null pointer dereference, array index out of bound, and class cast exceptions in unseen code.
• The interval-based propagation mechanism (IBPM) significantly improves GNN generalization, enabling better detection of complex, non-syntactic bug patterns.
• Despite improvements, NeurSA struggles with very large programs (~1,000 nodes), where local bug patterns are obscured by irrelevant code, indicating a need for better signal localization.
• False positives in NeurSA are often caused by the same issues that lead to false negatives—particularly over-sensitivity to null pointer presence without full context.