[Paper Review] GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search

Shangqing Liu, Xiaofei Xie+4 more|arXiv (Cornell University)|Nov 4, 2021

Software Engineering Research36 references4 citations

TL;DR

GraphSearchNet proposes a graph-based deep learning framework that enhances GNNs for semantic code search by jointly modeling source code and natural language queries through bidirectional GNNs (BiGGNN) and multi-head attention to capture both local structural and global dependencies. The model achieves state-of-the-art performance on CodeSearchNet benchmarks for Java and Python, significantly outperforming prior methods.

ABSTRACT

Code search aims to retrieve accurate code snippets based on a natural language query to improve software productivity and quality. With the massive amount of available programs such as (on GitHub or Stack Overflow), identifying and localizing the precise code is critical for the software developers. In addition, Deep learning has recently been widely applied to different code-related scenarios, e.g., vulnerability detection, source code summarization. However, automated deep code search is still challenging since it requires a high-level semantic mapping between code and natural language queries. Most existing deep learning-based approaches for code search rely on the sequential text i.e., feeding the program and the query as a flat sequence of tokens to learn the program semantics while the structural information is not fully considered. Furthermore, the widely adopted Graph Neural Networks (GNNs) have proved their effectiveness in learning program semantics, however, they also suffer the problem of capturing the global dependencies in the constructed graph, which limits the model learning capacity. To address these challenges, in this paper, we design a novel neural network framework, named GraphSearchNet, to enable an effective and accurate source code search by jointly learning the rich semantics of both source code and natural language queries. Specifically, we propose to construct graphs for the source code and queries with bidirectional GGNN (BiGGNN) to capture the local structural information of the source code and queries. Furthermore, we enhance BiGGNN by utilizing the multi-head attention module to supplement the global dependencies that BiGGNN missed to improve the model learning capacity. The extensive experiments on Java and Python programming language from the public benchmark CodeSearchNet confirm that GraphSearchNet outperforms current state-of-the-art works.

Motivation & Objective

• To address the challenge of semantic code search in the 'Big Code' era, where developers rely heavily on code reuse.
• To overcome the limitations of sequential models and standard GNNs in capturing both structural and global dependencies in code and queries.
• To improve semantic mapping between source code and natural language queries by jointly modeling their representations using graph neural networks.
• To enhance GNNs by integrating multi-head attention to recover global dependencies missed by conventional GNNs.

Proposed method

• Constructing program and query graphs using bidirectional Gated Graph Neural Networks (BiGGNN) to learn local structural information from code and query sequences.
• Enhancing BiGGNN with a multi-head self-attention mechanism to capture long-range and global dependencies across nodes in the graph.
• Jointly encoding source code and natural language queries into unified vector representations using the enhanced graph network.
• Computing semantic similarity between code and query embeddings using cosine similarity for retrieval.
• Training the model end-to-end on the CodeSearchNet dataset using contrastive learning objectives to align relevant code-query pairs.
• Applying graph construction techniques that include AST-based edges and token-level connectivity to preserve syntactic and semantic structure.

Experimental results

Research questions

• RQ1 Can a graph-based deep learning framework effectively model both local structural and global dependencies in source code and natural language queries for code search?
• RQ2 How does incorporating multi-head attention into BiGGNN improve the representation learning capacity of GNNs for code search?
• RQ3 To what extent does GraphSearchNet outperform existing state-of-the-art models in semantic code search on standard benchmarks?
• RQ4 Does the joint modeling of code and query graphs lead to better semantic alignment than sequential or unimodal approaches?

Key findings

• GraphSearchNet achieves state-of-the-art performance on the CodeSearchNet benchmark for both Java and Python programming languages.
• The model significantly outperforms existing SOTA methods, including MMAN and Code2Seq, in terms of mean average precision (mAP) and recall at top-k results.
• The integration of multi-head attention into BiGGNN leads to a measurable improvement in capturing global dependencies, especially in long-range structural relationships.
• The ablation study confirms that both BiGGNN and multi-head attention components are essential, with each contributing significantly to the final performance gain.
• GraphSearchNet demonstrates robust generalization across different programming languages, indicating its effectiveness beyond language-specific patterns.